Inside FinRAG

The "T" in ChatGPT wasn't invented by OpenAI

In 2017, eight Google researchers published "Attention Is All You Need" and introduced the Transformer, the T in GPT. Here is how attention works in plain English, how it led to ChatGPT, and why it now reads markets in seconds.

3 min read
The "T" in ChatGPT wasn't invented by OpenAI. (hero cover)
In this article

Most people met modern AI in November 2022, when ChatGPT launched. But the idea that powers it was published five years earlier, and not by OpenAI.

Watch the explainer video

June 2017: a short paper with a strange title

On 12 June 2017, eight researchers at Google posted a paper called "Attention Is All You Need". The title is a nod to the Beatles' "All You Need Is Love", as one of the authors has said.

The paper introduced a new design for AI language models: the Transformer.

How AI read text before

Earlier language models read text the way we do: one word at a time, left to right. By the end of a long sentence, they had partly lost track of the beginning. They were also slow to train, because each word had to wait for the one before it.

What attention does

The Transformer reads every word at once. Then, for each word, it asks a simple question: which other words matter to me?

Take the sentence: "The bank raised interest rates."

Attention links "bank" strongly to "rates". So the model understands it is a bank for money, not a river bank. The same word gets its meaning from the words around it, all computed at the same time.

Why it changed everything

Because every word is processed in parallel, the Transformer is a perfect fit for modern chips (GPUs), which do thousands of calculations at once. The paper's model reached state-of-the-art translation results after training for 3.5 days on eight GPUs.

Faster training meant much bigger models could be trained on much more text. That is the road to today's AI.

From Google's paper to ChatGPT

YearWhat happened
2017Google researchers publish the Transformer
2018OpenAI builds on it and names its model GPT: Generative Pre-trained Transformer
2022ChatGPT launches

All eight authors later left Google, many to start their own AI companies, as Wired reported. The paper has been cited well over 150,000 times, making it one of the most cited scientific papers of this century.

It reached Wall Street too

Today, transformer models read news, earnings call transcripts and company filings in seconds. They pull out what matters, compare it with past data and flag changes, work that used to take analysts hours. Many AI agents for trading and research are built on this same idea.

The lesson

The biggest breakthroughs often hide in plain sight, in a research paper most people never read.

So next time you hear "AI changed everything", remember: it started with one word. Attention.

References


Narration in the video is an AI voice (ElevenLabs). The light-web image is an AI-generated illustration. Citation counts vary by database and are approximate. FinRAG.tech has no connection to Google or OpenAI. This article is educational, not investment advice.

Worth sharing?

Pass this article along to someone who would find it useful.

The work continues in the studio.

More build notes, walkthroughs, and perspectives from the FinRAG team.

Explore all stories

A little more perspective

Keep exploring the studio.