Heterogeneous treatment effects (HTE)

Heterogeneous treatment effects describe how the true effect of a treatment varies across different subgroups of a population, rather than the single number an average treatment effect reports for everyone combined. A pricing change might increase revenue among price-insensitive users while reducing it among price-sensitive ones; the ATE could show a small net effect while hiding two much larger effects pulling in opposite directions.

Estimating this well is harder than estimating the ATE, because splitting an experiment into many subgroups quickly runs into small sample sizes per group and a multiple-comparisons problem: testing enough subgroup cuts will eventually surface a "significant" difference by chance alone. Purpose-built methods such as causal forests and other tree-based or model-based approaches estimate a conditional average treatment effect (CATE) for each unit or segment while trying to control this inflation, rather than testing every subgroup cut a human happens to think of.

HTE analysis matters for turning an experiment result into a targeting decision: rather than a binary launch-or-don't-launch call based on the ATE, a team can roll a treatment out only to the segments it actually helps. It underpins uplift modeling, and its main pitfall remains the one that affects any subgroup analysis, mistaking noise in a small slice of the data for a genuine, replicable difference.