Train-test split

A train-test split divides a dataset into two separate portions before modeling begins: a training set, used to fit the model, and a test set, held back and used only afterward to evaluate how the model performs on data it has never seen. A common split is 70-80% training and 20-30% test, though the right proportions depend on dataset size.

This separation exists because evaluating a model on the same data it was trained on overstates how well it will perform in the real world; a sufficiently flexible model can fit training data almost perfectly while badly overfitting to it, a gap that only shows up when tested on unseen data. Many projects add a third portion, a validation set, used during development to tune settings, keeping the test set completely untouched until a final, honest evaluation.

A single train-test split gives one estimate of performance, which can be noisy, especially with a small sample size; cross-validation addresses this by repeating the split-and-evaluate process multiple times and averaging the result. For data with a time dimension, such as sales forecasts, the split should respect chronological order, training on the past and testing on the future, rather than a random split that would let the model see the future during training.