Generative AI & LLMs
Large language models, prompting, retrieval (RAG), fine-tuning and image generation.
Expert
Research-level depth and specialist material.
Video
Expert
▶ Watch here
Let's Build GPT: From Scratch, in Code, Spelled Out
Karpathy writes a small GPT in PyTorch line by line, following the 'Attention Is All You Need' design.
Andrej Karpathy
Video
Expert
▶ Watch here
Let's Build the GPT Tokenizer
Builds a byte-pair-encoding tokenizer from scratch and explains many odd LLM behaviours that come from tokenisation.
Andrej Karpathy