Random forest

A random forest is an ensemble of many decision tree models, each trained on a different random sample of the training rows and a random subset of features considered at each split. Predictions from all trees are combined by averaging, for regression, or by majority vote, for classification.

Combining many trees built on different data and feature subsets reduces the variance that makes a single decision tree unstable: individual trees may overfit their particular sample, but their errors tend not to be correlated, so averaging cancels much of that noise out. This makes random forests markedly more resistant to overfitting than a lone tree, generally at the cost of being harder to interpret directly.

Random forests are a common default choice for tabular prediction problems because they require relatively little feature engineering, handle nonlinear relationships and interactions well, and provide a built-in measure of feature importance. Their main limitations are that they can be slow and memory-heavy at large scale, they do not extrapolate well beyond the range of training data, and they are usually outperformed on structured data by well-tuned gradient boosting models, though at greater tuning effort.