Glossary

Decision tree

A supervised model that predicts an outcome by following a sequence of if-then splits on feature values, shaped like a tree.

Also called: CART

A decision tree is a supervised learning model that predicts an outcome by repeatedly splitting data on the feature value that best separates the target, forming a tree of yes/no questions. Each internal node tests one feature, for example "is annual spend over $500," and each leaf holds a prediction: a class label for classification, or an average value for regression.

Splits are chosen to reduce impurity in the resulting groups, using measures such as Gini impurity or information gain for classification, and variance reduction for regression. A tree can, in principle, keep splitting until every leaf is pure, but an unconstrained tree memorizes the training data and generalizes poorly; practitioners limit this with maximum depth, a minimum number of samples per leaf, or pruning after the fact.

Decision trees matter because their logic is easy to read and explain to non-technical stakeholders, unlike many other models. In practice a single tree is rarely used alone for its predictive accuracy: it is the building block behind random forest and gradient boosting models, which combine many trees to reduce the overfitting and instability a single tree is prone to, at the cost of that easy interpretability.

Last reviewed September 22, 2026

In the index now

Related terms

Random forest An ensemble model that trains many decision trees on random subsets of data and features, then averages their predictions. Gradient boosting An ensemble method that builds trees one at a time, each one correcting the errors made by the trees before it. Classification A supervised learning task that assigns each input to one of a fixed set of discrete categories or labels. Overfitting When a model learns the noise and specific quirks of its training data so closely that it performs poorly on new data. Feature engineering Creating, transforming or selecting the input variables a model is trained on, to make the underlying pattern easier for it to learn.