Deep Learning Learning

Perwork - Math

• 3B1B Essence of Linear Algebra
• 3B1B Essence of Calculus
• 3B1B Essence of Probability
• 3B1B Essence of Neural Networks

Readings

• Unreasonable Effectiveness of RNN
• Colah's Blog

Foundation - Deep Learning

• Andrej Karpathy's Zero to Deep Learning
• Deep Learning Book
• Transformer Explainer

Coursera For NNs

• NNs
• CNNs
• RNNs + Transformer Intro
• PyTorch

UT Austin Masters Courses

• Advanced Linear Algebra
• Deep Learning

Core Papers

• AlexNet
• Deep Residual NN
• Sequence to Sequence Learning with Neural Networks
• Neural Machine Translation

---

• Attention Is All You Need
• BERT
• GPT-2
• GPT-3
• Scaling Laws for NLP
• Chinchilla

---

• InstructGPT
• DPO for LLM
• GPT-4 Technical Report

---

• Chain of Thought
• Tree of Thought
• Voyager
• Generative Agents

---

• Mixture of Experts
• LoRA
• Learning Transferable Visual Models From Natural Language Supervision
• Flamingo: a Visual Language Model for Few-Shot Learning
• Visual Instruction Tuning

---

• DeepSeek-V3 Techinal Report
• OpenAI o1 System Card
• Absolute Zero: Reinforced Self-play Reasoning with Zero Data
• The Llama 3 Herd of Models
• Gemini 1.5

---

• Flash Attention
• RAG
• Alpha Evolve