Correlation, intervention, counterfactual: the three levels every AI system lives on

Ask an AI system a question and, underneath the answer, it is operating at one of three levels. The levels form a hierarchy — each one answers a harder question than the last, and each demands more than the one below. Almost all of today’s AI is stuck on the bottom level. Almost every decision that matters requires the top one. The gap between those two facts is the single most important thing to understand about where AI can and can’t be trusted.

The three levels

The hierarchy is often called the ladder of causation, and it separates three kinds of question that look similar but are worlds apart.

The three levels of causal reasoning: association (seeing), intervention (doing), counterfactual (imagining).
Figure 1 — Almost all of today’s AI lives on the first rung. Almost every real decision needs the third.

Level 1 — Association. “Seeing.” What does X tell me about Y? This is the level of correlation and pattern-matching. When you observe one thing, how should you update your belief about another? Nearly all of modern machine learning lives here: deep networks, forecasters, recommender systems, and large language models are, at their core, extraordinarily powerful engines for the probability of Y given that you’ve seen X. It is a level of pure observation. The world is watched, never touched.

Level 2 — Intervention. “Doing.” What happens if I set X? Now you’re not observing X, you’re acting on it — reaching in and changing it deliberately. This is a different question, and the answer is usually different too. Observing that X and Y move together tells you little about what happens when you force X to a new value. This is the territory of A/B tests, reinforcement learning, and every policy or business decision, all of which are really asking: if we do this, what follows?

Level 3 — Counterfactual. “Imagining.” What would have happened if I’d done otherwise? This is the most demanding question of all — a claim about a world that never occurred. Given what actually happened, you rewind and imagine a different choice. It’s the level of attribution (“did the drug cause the recovery, or would she have recovered anyway?”), of credit and blame, and of individualized effect. It requires not just knowing the effect of an action in general, but reconstructing the specific alternate history of a specific case.

Why the levels don’t collapse into each other

The crucial, counterintuitive fact is that you cannot answer a higher-level question with lower-level tools — no matter how much data you have.

A perfect model of association — one that predicts flawlessly — still cannot tell you what happens when you intervene. The classic trap: ice-cream sales and drownings rise together, so each perfectly predicts the other. But intervening to ban ice cream saves no one, because summer heat drives both. A Level-1 model, however accurate, answers a Level-2 question wrongly, and can’t tell that it’s doing so.

Here’s the same failure with a decision attached.

A coupon example answered at three levels: plus 30 percent (association, confounded), plus 8 percent (intervention, the real effect), and zero for a customer who would have bought anyway (counterfactual).
Figure 2 — All three answers are “correct.” Only one fits each decision.

A store notices that customers who received a discount coupon spent 30% more (Level 1). Tempting — but coupons went to already-loyal buyers, so that number is confounded. The actual effect of sending coupons, measured by intervening, is a lift of 8% (Level 2). And for one specific loyal customer who would have bought anyway, the coupon changed nothing at all — pure giveaway (Level 3). Three answers, all correct, each belonging to a different question. Set your marketing budget using the Level-1 number and you’ll massively overspend, confidently, on the strength of a real statistic answering the wrong question.

What it takes to climb

If more data doesn’t lift you up the ladder, what does? A causal model — a representation of not just what correlates with what, but what causes what. The direction of the arrows. The mechanism.

This is why the levels matter so much for AI right now. The systems capturing the most attention — large models trained to predict the next token or the next frame — are Level-1 systems of staggering ability. They are matchless at seeing. But seeing is not doing, and doing is not imagining. The moment you ask one of these systems to plan, to act, to be held accountable for a decision — the moment it becomes agentic — you are asking a Level-2 or Level-3 question, and a Level-1 engine cannot answer it reliably no matter how large it grows.

That is the gap. Not a gap in scale or data, but a gap in kind. Climbing it requires building the thing a predictive model doesn’t have: a model of consequence.

The bottom line

Three questions, three levels. What tells me about what. What happens if I act. What would have happened otherwise. Prediction — the thing modern AI does so well — is the first and lowest of the three. Every decision worth making, and every action worth taking, lives higher up.

Knowing which level a question belongs to — and refusing to answer it with tools from a lower one — is the difference between an AI that describes the world and one you can actually act on.