Reasoning Models and Test-Time Compute

For most of the history of machine learning, compute was something you spent during training. Once a model was trained, inference — generating an answer — was fast and cheap. The model would receive a prompt and produce a response in a single forward pass through its billions of parameters, typically in seconds. This forward-pass cost was roughly fixed regardless of how hard the question was. A simple greeting and a PhD-level mathematics problem cost essentially the same amount of compute to answer. In 2024, a new class of models changed this picture fundamentally. Reasoning models — exemplified by OpenAI's o1 and o3, and Anthropic's Claude models with extended thinking — deliberately spend more compute at inference time on harder problems. They think before they answer. The harder the problem, the longer they think. This approach, called test-time compute scaling, represents one of the most significant architectural shifts in frontier AI in recent years.

What Test-Time Compute Means

Test time refers to the moment when a trained model is used — when you give it a prompt and it generates a response. Training time is when the model learns from data; test time is when it applies what it learned. For most models, the computation at test time is a single deterministic pass through the network: input goes in, output comes out. Test-time compute scaling means allocating additional computational resources at inference. There are several mechanisms for doing this. The simplest is generating multiple candidate answers and selecting the best one — a technique called best-of-N sampling. If you generate 64 answers to a math problem and pick the one that passes the most internal consistency checks, your answer quality improves even if the underlying model is unchanged. More powerful is the approach used by reasoning models: a trained internal deliberation process. The model generates an extended chain of thought — sometimes hundreds or thousands of tokens of working — before producing a final answer. This working is often not shown to the user; it is the model's internal scratchpad. The key insight is that the model was trained to use this scratchpad effectively, via reinforcement learning on problems where the correct answer can be verified. By rewarding the model when its deliberation leads to correct answers, training teaches it to use the scratchpad for genuinely useful computation rather than filler text.

The o1 and Extended Thinking Paradigm

OpenAI's o1 model, released in September 2024, demonstrated that training a model to think before answering — via reinforcement learning on verifiable problems — produced dramatic improvements on hard reasoning benchmarks. On the American Invitational Mathematics Examination (AIME), a competition-level exam where GPT-4o scored around 12%, o1 scored 74%. On a PhD-level science benchmark called GPQA Diamond, o1 exceeded the average performance of domain experts. These are not incremental improvements; they represent qualitatively different performance on tasks that were previously considered very difficult for AI. The key to o1's design is that its extended reasoning is trained, not just prompted. Earlier chain-of-thought prompting asked the model to show its work; o1 was reinforcement-trained to generate reasoning traces that actually lead to correct answers on problems where ground truth can be verified — math, coding, and formal logic. This is a subtle but crucial distinction: the model learned that certain patterns of internal deliberation improve its accuracy, because it was explicitly rewarded for correct answers rather than for producing plausible-sounding reasoning. Anthropic's extended thinking mode works on a similar principle: Claude is given a reasoning budget and uses it to deliberate — exploring solution paths, checking consistency, and revising conclusions — before delivering its final answer. The visible 'thinking' output represents genuine intermediate computation, not post-hoc rationalization.

Trade-offs and Limitations

Test-time compute scaling is not a free lunch. The most obvious cost is latency and expense: a model that thinks for thirty seconds before answering uses substantially more compute than one that answers in two seconds. For interactive applications where users expect fast responses, this is a real constraint. For high-stakes tasks — medical diagnosis support, legal reasoning, safety-critical engineering calculations — the latency cost is often worth paying. A subtler concern is faithfulness: does the visible chain of thought actually represent what the model is doing, or is it a post-hoc narrative? Research suggests the chain of thought is causally connected to the output — perturbing it changes the answer — but the model may also produce plausible-sounding reasoning that obscures the true computational path. Interpretability in reasoning models remains an open problem. Finally, test-time compute helps most on problems with verifiable ground truth — math, code, logic puzzles — where reinforcement learning can assign clear rewards. On problems requiring subjective judgment, creative synthesis, or ethical reasoning, the benefit is harder to measure and the training signal is noisier.