Building Reasoning Models from the Ground Up: A Video Course

Introduction

If you’ve ever wanted to see how modern reasoning models work under the hood, the new Build a Reasoning Model (From Scratch) video course is a hands‑on guide that takes you from a pre‑trained LLM to a fully‑featured reasoning engine. Over eight lectures (more than 11 hours of content) you’ll implement evaluation, inference‑time scaling, reinforcement learning, and distillation in Python with PyTorch, all while learning the practical trade‑offs that matter in real projects.

---

1. From a Base Model to a Reasoning Engine

The series begins by loading a small pre‑trained Qwen‑3 model and establishing a solid text‑generation pipeline. You’ll learn how to:

• Encode and decode tokens, run greedy decoding, and stream output efficiently.
• Add a key‑value (KV) cache and use `torch.compile` to accelerate inference.
• Set up the development environment (Python, PyTorch, uv, GPU/CPU considerations) so you can reproduce the experiments on modest hardware.

These foundations give you a baseline model that you can benchmark before any reasoning‑specific enhancements.

---

2. Building Verifiers and Reward Signals

A core piece of any reasoning model is a reliable way to judge its answers. In lecture three you’ll construct a math‑answer verifier that:

• Extracts final answers from generated text.
• Normalizes and checks mathematical equivalence.
• Provides a deterministic reward signal for reinforcement learning (RLVR).

The same verifier is used to evaluate the base model on the MATH‑500 benchmark, establishing a clear performance metric for later improvements.

---

3. Inference‑Time Scaling Techniques

Two consecutive lectures explore how to squeeze more accuracy out of a fixed model:

Sampling, Temperature, and Top‑p

You’ll implement temperature scaling, multinomial sampling, and nucleus (top‑p) filtering to generate diverse answer candidates. By aggregating these candidates with self‑consistency (majority voting), you can boost accuracy without changing model weights.

Scoring and Self‑Refinement

Next, you’ll compute token‑level log probabilities and sequence likelihoods to rank candidates and break ties. A self‑refinement loop lets the model critique its own answer and produce a revised response, further improving results on the MATH‑500 set.

Both sections include cost‑vs‑benefit analyses, showing how additional compute translates into higher accuracy.

---

4. Reinforcement Learning with Verifiable Rewards (RLVR)

The final technical deep‑dive covers Group Relative Policy Optimization (GRPO), a simplified RL algorithm that uses the verifier’s scores as rewards. You’ll:

• Generate groups of reasoning traces.
• Convert verifier scores into relative advantages.
• Update model weights based on these advantages, contrasting RLVR with the more common RLHF approach.

By the end of this lecture you’ll have a working RL loop that can fine‑tune the model toward higher‑quality reasoning.

---

Practical Takeaways

• Hands‑On Code – Every concept is paired with runnable PyTorch snippets, so you can experiment immediately.
• Modular Learning Path – The videos work as a sequel to the earlier Build a Large Language Model course but stand alone for newcomers.
• Resource Guidance – Early chapters run on a CPU; later RL and distillation steps benefit from a GPU, and the repository includes hardware recommendations.
• Complementary Materials – A companion book (available from Manning/Amazon) and a short 90‑minute LinkedIn Learning overview provide additional context and exercises.

---

Closing Thoughts

Reasoning models are the engine behind today’s AI agents, and understanding their internals is essential for anyone building advanced LLM‑powered applications. This video series offers a complete, code‑first pathway—from loading a base model to applying sophisticated inference scaling and reinforcement‑learning techniques. Whether you’re a researcher, developer, or hobbyist, working through the lectures and accompanying book will give you a concrete foundation for creating, evaluating, and improving reasoning models in real‑world settings.