State of GPT

State of GPT is an educational session/talk delivered by Andrej Karpathy at Microsoft Build 2023, hosted on build.microsoft.com. It covers the full GPT assistant training pipeline — from tokenization and pretraining through supervised fine-tuning, reward modeling, and RLHF — alongside practical techniques for effectively applying these models in real-world applications. The session also explores prompting strategies, retrieval augmentation, fine-tuning, plugins, and the broader LLM ecosystem, making it accessible to both newcomers and experienced practitioners.