$\newline Logo$

Go to Preview Lesson

Go to Preview Lesson

LESSON 5.1N-Gram Language Models (Mini Project 3)

LESSON 5.2Triplet Loss Embedding Finetuning for Search & Ranking (Mini Project 4)

MODULE 1
Onboarding & Tooling
MODULE 2
AI Projects and Use Cases
MODULE 3
Prompt Engineering & Embeddings
- LESSON 3.1Prompt Engineering — From Structure to Evaluation (Mini Project 1)
- LESSON 3.2Tokens, Embeddings & Modalities — Foundations of Understanding Text, Image, and Audio
MODULE 4
Multimodal + Retrieval-Augmented Systems
- LESSON 4.1Multimodal Embeddings (CLIP)
- LESSON 4.2RAG & Retrieval Techniques (Mini Project 2)
MODULE 5
Classical Language Models
- LESSON 5.1N-Gram Language Models (Mini Project 3)
- LESSON 5.2Triplet Loss Embedding Finetuning for Search & Ranking (Mini Project 4)
MODULE 6
Attention & Finetuning
- LESSON 6.1Building Self-Attention Layers
- LESSON 6.2Instructional Finetuning with LoRA (Mini Project 5)
MODULE 7
Architectures & Multimodal Systems
- LESSON 7.1Feedforward Networks & Loss-Centric Training
- LESSON 7.2Multimodal Finetuning (Mini Project 6)
MODULE 8
Assembling & Training Transformers
- LESSON 8.1Full Transformer Architecture (From Scratch)
- LESSON 8.2Advanced RAG & Retrieval Methods
MODULE 9
Specialized Finetuning Projects
- LESSON 9.1CLIP Fine-Tuning for Insurance
- LESSON 9.2Math Reasoning & Tool-Augmented Finetuning
MODULE 10
Advanced RLHF & Engineering Architectures
- LESSON 10.1Preference-Based Finetuning — DPO, PPO, RLHF & GRPO
- LESSON 10.2Building AI Code Agents — Case Studies from Copilot, Cursor, Windsurf
MODULE 11
Agents & Multimodal Code Systems
- LESSON 11.1Agent Design Patterns
- LESSON 11.2Text-to-SQL and Text-to-Music Architectures
MODULE 12
Deep Internals & Production Pipelines
- LESSON 12.1Positional Encoding + DeepSeek Internals
- LESSON 12.2LLM Production Chain (Inference, Deployment, CI/CD)
MODULE 13
Enterprise LLMs, Hallucinations & Career Growth
MODULE 14
AI Accelerator
- LESSON 14.1AI Accelerator
MODULE 15
Career Prep
- LESSON 15.1Career Prep

Go to Next Lesson

Go to Next Lesson

LESSON 6.1Building Self-Attention Layers

Go Pro
Log In

Triplet Loss Embedding Finetuning for Search & Ranking (Mini Project 4)

- Triplet-Based Embedding Adaptation - User-to-Music & E-commerce Use Cases

Courses
AI Accelerator
Triplet Loss Embedding Finetuning for Search & Ranking (Mini Project 4)

Go To Previous Lesson
N-Gram Language Models (Mini Project 3)
- Understand what n-grams are and how they model language with simple probabilities - Implement bigram and trigram extraction using sliding windows over character sequences - Construct frequency dictionaries and normalize into probability matrices - Sample random text using bigram and trigram models to generate synthetic sequences - Evaluate model quality using entropy, character diversity, and negative log likelihood (NLL) - One-hot encode inputs and build PyTorch models for bigram and trigram neural networks - Train models with cross-entropy loss and monitor training dynamics - Compare classical vs. neural models in terms of coherence, prediction accuracy, and generalization
Go To Next Lesson
Building Self-Attention Layers
- Understand the motivation for attention: limitations of fixed-window n-gram models - Explore how word meaning changes with context using static vs contextual embeddings (e.g., "bank" problem) - Learn the mechanics of self-attention: Query, Key, Value, dot products, and weighted sums - Manually compute attention scores and visualize how softmax creates probabilistic context focus - Implement self-attention layers in PyTorch using toy examples and evaluate outputs - Visualize attention heatmaps using real LLMs to interpret which words the model attends to - Compare loss curves of self-attention models vs trigram models and observe learning dynamics - Understand how embeddings evolve through transformer layers and extract them using GPT-2 - Build both single-head and multi-head transformer models; compare their predictions and training performance - Implement a Mixture-of-Experts (MoE) attention model and observe gating behavior on different inputs - Evaluate self-attention vs MoE vs n-gram models on fluency, generalization, and loss curves - Run meta-evaluation across all models to compare generation quality and training stability