Generative Models: What They Are, How They Work and Where They Came From
The Climb of Information
An interactive piece I made: one continuous ascent from the Big Bang to AI — five eras and twenty-three rungs, tracing how information has been stored, copied, processed and represented at ever-higher levels, each step paid for in energy and echoed in computing. Open the deck → (use ← / → or space to move; best on a larger screen.)
Core Learning Resources
- 3Blue1Brown Neural Networks Series – YouTube video series on neural networks
- 3Blue1Brown Linear Algebra Series – Linear algebra foundation series
- Ars Technica: Jargon-Free LLM Explanation – How AI large language models work
- Foundations of Large Language Models – arXiv research paper
- Welch Labs: Visual AI Introduction
- Welch Labs: DeepSeek Transformer Analysis
- DeepLearning.AI: How Transformer LLMs Work – Free course (registration required)
- Distill Series on Pre-ChatGPT Evolution – Visual summaries of AI evolution
- The Illustrated Transformer – Jay Alammar
- Transformer Explanation Video – Using Jay Alammar's explainer
- Notebook LM "Podcast" on Attention Paper – AI-generated podcast on "Attention is All You Need"
- The Moment We Stopped Understanding AI (AlexNet)
- Andrej Karpathy: The Busy Person's Intro to LLMs
- Why Machines Learn: The Elegant Maths Behind Modern AI – Anil Ananthaswamy (Book)
- Helen Toner's LLM Explainer – "Next word prediction" in LLMs
- Anthropic Interpretability Research
- Simon Willison: Model Capability Assessment
- The Bitter Lesson – Scaling and compute in AI
- Gwern on Scaling
Resources for a Deeper Dive
- Michael Nielsen's Neural Networks eBook – Free online ebook (recommended by 3Blue1Brown)
- Andrej Karpathy's CS231n Course – Stanford University, Winter 2016 (start with lecture 2)
- Deep Learning Textbook – Free online version
- Stanford Deep Learning Tutorial
- Ilya Sutskever's AI Reading List – For John Carmack
How to Use Them?
- Andrej Karpathy's LLM Use Cases Guide
- March 2025 Model Capabilities Review – "Opinionated guide on which AI model"
- Gemini Quality Assessment – "Why Gemini is actually very good"
- Land Use Research with Deep Research – Successful practical application example
- Economics Research with Deep Research – Economist's successful use case
- Tyler Cowen's Model Usage – How different models are used
What Are the Dangers of Using Them?
- MIT Cognitive Debt Study – "Better work product, but you become less smart because you didn't do the work"
Personal Favorite Use Cases
Parallel Translation Reading: Using Claude on mobile for non-English texts. Live translations often superior to published versions. Side-by-side paragraph reading.