How Generative AI Models Work — Inside the Engine
In Lesson 1, you learned what Generative AI is. Now we go deeper: How does a model actually understand text? How does it learn? How does it generate a response? And why do these models require enormous computing resources? This lesson answers all of that — clearly, without requiring a mathematics or computer science background.
🎯 What You Will Learn in This Lesson
- What a Neural Network is and how it processes information
- What Deep Learning is and how it differs from standard Machine Learning
- What a Transformer is and why it revolutionized AI
- What a Token is and how models break down text
- What Embeddings are and how meaning is represented mathematically
- What Context means in language models
- How models predict the next word (and why this matters)
- What an LLM is and how it differs from Generative AI broadly
- What Parameters are and what model size really means
- The difference between Pre-training and Fine-tuning
- Why AI models need GPUs and massive computing infrastructure
What is a Neural Network?
An artificial neural network is a computational system loosely inspired by the structure of the human brain — not a precise simulation of it, but an adaptation of one key idea: process information through multiple connected layers of units.
Each unit in the network is called an artificial neuron. It receives inputs, applies a mathematical function, and passes the result to the neurons in the next layer. These connected layers form a pipeline through which data flows and transforms.
What makes neural networks powerful is their capacity to learn. When the network produces an incorrect output, the strength of connections between neurons is automatically adjusted. This correction process — called backpropagation — happens millions of times during training, gradually improving the network's accuracy.
What is Deep Learning?
Deep Learning refers to neural networks with many stacked layers — the "depth" refers to the network's structure, not complexity of the concept.
What does depth give you? The ability to learn extremely complex patterns that can't be captured by simple rules. In traditional Machine Learning, engineers manually design features — they tell the model what to look for. In Deep Learning, the model discovers those features on its own through its layers.
What is a Transformer and Why Does It Matter?
A Transformer is a specific neural network architecture introduced in a 2017 research paper titled "Attention is All You Need," published by researchers at Google. Since then, it has become the foundation of virtually every major Generative AI model, including GPT, Gemini, Claude, and Llama.
What Problem Did the Transformer Solve?
Previous architectures struggled with a fundamental challenge: understanding the relationships between words that are far apart in a sentence.
Consider: "The bank that we built from stones is too small." The word "built" refers to "bank" — but they're separated by other words. Older models often failed at this kind of long-range dependency. The Transformer solved it with a mechanism called Self-Attention.
What is Self-Attention?
Self-Attention is a mechanism that allows the model to determine how relevant each word is to every other word when processing a sentence. When processing any given word, the model "looks at" all other words in the context and assigns each one a relevance score.
The result: Transformer-based models understand long-range context and complex language relationships far better than anything that came before.
What is a Token?
Models don't read text the way humans do — letter by letter or word by word. Instead, they break text into smaller units called Tokens.
A Token might be:
- A whole word: "generative"
- Part of a word: "genera" + "tive"
- A punctuation mark: "." or ","
- A space combined with a word
For example, the sentence "Generative AI is powerful" might be tokenized as:
What is an Embedding?
Computers can't process words directly. For a model to work with text, it converts each Token into a numerical vector — a list of numbers — called an Embedding.
The key insight in Embeddings: words with similar meanings have vectors that are close to each other in mathematical space. "King" and "queen" are closer to each other than "king" and "apple."
In modern Generative AI, Embeddings represent not just individual words but entire sentences, paragraphs, and complex concepts. This is the foundation of RAG (Retrieval-Augmented Generation), a technique covered in depth in Lesson 5.
What is Context in a Language Model?
Context is everything the model can "see" when generating a response — the conversation history, any instructions, attached documents, and the current question.
A language model has no persistent memory between separate conversations. Each time it generates a response, it processes everything within its Context Window and produces output based on that information alone.
When you ask ChatGPT a follow-up question that refers to something mentioned earlier in the conversation, the model responds coherently because the entire conversation exists within its Context Window at that moment. If the conversation exceeds the Context Window size, earlier content gets dropped — which is why very long conversations can cause models to "forget" what was discussed at the beginning.
How Does the Model Predict the Next Word?
The fundamental mechanism of language models is straightforward in principle: predict the next token.
When generating a response, the model starts from scratch. It takes everything in the Context Window, processes it through the Transformer layers, and produces a probability distribution across all possible tokens in the language. It then selects a token (the most probable, or a controlled random selection for variety) and adds it to the context. Then it repeats the process for the next token — until the response is complete.
What is an LLM? How Does It Differ from Generative AI?
LLM stands for Large Language Model. It is a specific category of Generative AI model trained on massive amounts of text data, specialized in understanding and generating human language.
| Aspect | Generative AI (broad) | LLM |
|---|---|---|
| Scope | Broad — covers text, images, audio, video | Narrower — specialized in language |
| Examples | DALL·E (images), Sora (video), Whisper (audio) | GPT-4, Gemini, Claude, Llama |
| Primary input | Varies by model type | Text (some multimodal models accept images too) |
| Common use | Diverse content creation | Conversation, summarization, analysis, coding |
Every LLM is a type of Generative AI, but not every Generative AI is an LLM. DALL·E generates images — it's Generative AI but not a language model.
What Are Parameters?
Parameters are the numerical values that define a model's behavior — they are what the model learns during training. Think of them as billions of finely tuned settings that determine how the model responds to any given input.
Some reference points on model scale:
- GPT-3: approximately 175 billion parameters
- GPT-4 and newer frontier models: estimated in the hundreds of billions to trillions (exact numbers are not publicly disclosed)
- Open-source models like Llama: range from 7 billion to 70+ billion parameters in various sizes
Pre-training vs. Fine-tuning
Pre-training
Pre-training is the first and largest phase of model development. The model is exposed to enormous general-purpose datasets and learns broad language patterns, world knowledge, and reasoning capabilities. This phase takes weeks or months, requires thousands of GPUs running continuously, and costs millions of dollars. Only well-funded organizations like OpenAI, Google, Meta, and Anthropic can train frontier models from scratch.
Fine-tuning
After pre-training, a model can be further trained on specialized data to improve its performance in a specific domain. Examples:
- Fine-tuning on legal documents → a model better suited for legal analysis
- Fine-tuning on customer support conversations → a more effective support chatbot
- Fine-tuning on medical literature → a model better equipped for healthcare applications
What is a GPU and Why Do AI Models Need So Much Computing Power?
GPU stands for Graphics Processing Unit. Originally designed for rendering graphics in video games, GPUs became the backbone of AI model training because of a critical architectural feature.
CPU vs. GPU for AI
A CPU processes complex tasks sequentially — one after another. It's excellent for intricate logic. A GPU, by contrast, can execute thousands of simple operations simultaneously. Training a neural network requires enormous numbers of straightforward matrix multiplications — exactly the kind of work GPUs excel at.
| Aspect | CPU | GPU |
|---|---|---|
| Cores | Typically 8–64 | Thousands (NVIDIA H100: ~16,000 cores) |
| Operation type | Complex, sequential tasks | Simple, massively parallel tasks |
| Role in AI | Orchestration and control logic | Training and running large models |
Why Is Training So Expensive?
Training a frontier model requires:
- Thousands of specialized GPUs running simultaneously
- Weeks or months of continuous operation
- Massive energy consumption
- Sophisticated cloud or on-premises infrastructure
This is why only large, well-capitalized organizations train foundation models from scratch. However, running inference (using an already-trained model) is significantly cheaper — which is what users pay for via monthly subscriptions to tools like ChatGPT or Gemini.
How the Model Responds to "What is the capital of the UAE?"
- Your text is split into tokens
- Each token is converted into an Embedding (numerical vector)
- Embeddings pass through the Transformer layers, which analyze relationships between tokens
- The model produces a probability distribution: "Abu Dhabi" scores highest, followed by other possibilities
- "Abu" is selected and added to the context
- "Dhabi" follows, then the rest of the sentence
- The process repeats until the response is complete
All of this happens in fractions of a second — enabled by GPU parallel processing.
Practical Exercise
Visit OpenAI's interactive Tokenizer at: platform.openai.com/tokenizer
Paste any text and observe how it's broken into tokens. Notice that uncommon words or technical terms are often split into multiple tokens. Try different languages and see how tokenization differs. This gives you a concrete feel for why context window size is measured in tokens, not words.
✅ Key Takeaways from Lesson 2
- Neural Networks process data through multiple layers of mathematical units and learn by adjusting connection weights
- Deep Learning uses many-layered networks to automatically learn complex patterns from data
- The Transformer architecture — specifically its Self-Attention mechanism — is what enables modern AI's language understanding
- Tokens are the basic units models process; they're not always full words
- Embeddings convert meaning into numerical vectors — similar concepts have numerically similar representations
- Context Window defines how much the model can "see" when generating a response
- Models generate text by predicting the most appropriate next token, one at a time
- LLMs are a specialized category of Generative AI focused on language
- Parameters are what models learn — quality and quantity both matter
- Pre-training builds general capability; Fine-tuning adapts it for specific domains
- GPUs are essential because AI training requires massive parallel mathematical computation
⚠️ Common Mistakes
- Thinking the model "knows" things the way a database does: Models don't retrieve stored facts — they generate responses based on learned patterns
- Confusing LLM with Generative AI: LLMs are specialized in language; Generative AI is broader
- Assuming bigger always means better: Training data quality and methodology matter as much as size
- Treating Fine-tuning as the default solution: Good Prompt Engineering (Lesson 3) often achieves excellent results without the cost and complexity of fine-tuning
Glossary
| Term | Definition |
|---|---|
| Neural Network | A computational system that processes data through connected layers of mathematical units |
| Deep Learning | Machine Learning using many-layered neural networks to learn complex patterns automatically |
| Transformer | A neural network architecture that uses Self-Attention to understand relationships between all parts of an input sequence |
| Self-Attention | A mechanism that scores the relevance of every token to every other token in the context |
| Token | The basic unit of text a model processes — may be a word, word fragment, or punctuation |
| Embedding | A numerical vector representation of a word or concept, where similar meanings map to nearby vectors |
| Context Window | The maximum number of tokens a model can process in a single pass |
| LLM | Large Language Model — a Generative AI model specialized in language understanding and generation |
| Parameters | The numerical values a model learns during training that determine its behavior |
| Pre-training | Initial large-scale training on general data to build broad language capabilities |
| Fine-tuning | Additional training on specialized data to improve performance in a specific domain |
| GPU | Graphics Processing Unit — specialized hardware that executes thousands of parallel computations, essential for AI training |