Prediction Tells You What. Counterfactuals Tell You What If.

A weather forecast tells you it will rain tomorrow. Useful. But it never has to answer the harder question: would it have rained if we hadn’t seeded the clouds? A sales forecast says revenue will climb next quarter. Also useful — until someone asks the question that actually decides the budget: would revenue have climbed anyway, without the campaign we’re about to pay for?

That second kind of question is a counterfactual — a claim about a world that didn’t happen. And it turns out that almost every decision worth making depends on one.

Three questions, not one

It helps to separate the questions a model can be asked to answer. They form a ladder, and each rung demands more than the last.

The bottom rung is seeing: what is likely to happen, given what we’ve observed? This is the home of correlation, prediction, and classical forecasting. When X rises, Y tends to rise, so we project Y forward. Powerful, but purely associative — it describes the world as it rolls along undisturbed.

The middle rung is doing: what happens if we intervene? Not “when X happens to be high, what’s Y?” but “if we deliberately set X to this value, what will Y do?” The distinction is not pedantic. Ice-cream sales predict drownings — but banning ice cream saves no one, because the summer heat drives both. Acting on a correlation as though it were a lever is how organizations waste fortunes.

The top rung is imagining: given what we actually did and saw, what would have happened if we’d done something else? This is the counterfactual — and it’s the only rung on which you can properly evaluate a decision after the fact, or choose between decisions before making one.

Forecasting, for all its sophistication, sits on the bottom rung. Decisions live at the top.

Why causal inference is counterfactual reasoning

Ask what the “effect” of an action even means, and you arrive at a counterfactual whether you like it or not.

The effect of a treatment on a patient is the difference between two outcomes: how they fared having taken the drug, and how the same patient would have fared without it. The effect of a policy is the gap between the world that unfolded and the world that would have unfolded under the status quo. In every case the causal effect is defined as a comparison between the factual outcome and a counterfactual one.

This is also where the whole enterprise gets hard. You can only ever observe one of those two worlds. The patient either took the drug or didn’t. The company either ran the campaign or didn’t. The other branch — the road not taken — is never measured. This is the fundamental problem of causal inference: the counterfactual is missing by construction.

So it has to be modeled. Everything in causal inference — randomized trials, matching, instrumental variables, structural models — is ultimately machinery for constructing a credible estimate of the outcome you didn’t get to see.

Forecasting, upgraded

Bring counterfactuals to forecasting and the discipline changes character. Instead of a single projected line, you reason about two: what actually happened, and what would have happened otherwise. The space between them is the thing you actually care about.

This is the logic behind causal-impact and policy-evaluation methods. You learn the system’s behavior before an intervention, project the counterfactual — the “if we’d done nothing” trajectory — forward through the post-intervention period, and read the effect off as the gap between that projection and what was actually observed. Did the new pricing lift retention, or was retention rising anyway? Did the regulation reduce emissions, or were they already falling? A plain forecast can’t separate the two. A counterfactual forecast is built to.

The same structure answers forward-looking decisions, not just backward-looking evaluation. “What if we changed this?” is a counterfactual about the future — a comparison between the trajectory under the action and the trajectory without it. That is scenario forecasting done honestly: not one prediction, but a contrast between possible worlds.

The catch, and why it demands causal structure

Here is the uncomfortable part. Because you never observe the counterfactual, you cannot validate it the way you’d validate a forecast — there is no held-out ground truth for a world that didn’t happen. The credibility of the counterfactual rests entirely on the model that produced it.

A model that has only learned correlations is on thin ice here. It can reproduce the past beautifully and still generate a nonsense counterfactual, because it has no representation of what causes what — only of what tends to co-occur. The moment you ask it about a deliberate intervention, it answers with a pattern from a world where no such intervention was made. It’s answering the wrong question and can’t tell.

A counterfactual is only as trustworthy as the causal structure behind it. To imagine the road not taken, a model needs to know how the system’s variables actually influence one another — the mechanism, not just the statistics. That’s the difference between a confident guess and an estimate a decision-maker can stand behind.

The bottom line

Forecasting asks what the future holds. Causal inference and counterfactual forecasting ask something more demanding and far more useful: what difference will our choice make? The first describes the world. The second lets you act on it.

Every real decision is a bet on a counterfactual — that doing this rather than that will change the outcome. A model that can reason about the roads not taken is a model you can actually decide with.