Glossary

Cross-validation

A method for estimating how well a model generalizes by repeatedly training and testing it on different splits of the same data.

Also called: k-fold cross-validation, CV

Cross-validation estimates how well a model will perform on new data by splitting the available data into several parts, training the model on some parts and testing it on the held-out part, then repeating this across different splits. The most common form, k-fold cross-validation, divides the data into k equal folds, trains on k-1 of them and tests on the remaining fold, k times, so every observation is used for testing exactly once; reported performance is the average across folds.

This differs from a single train-test split, which holds out one fixed portion of data and gives one performance estimate that can vary a lot depending on which rows happened to land in the test set. Cross-validation instead produces several estimates, giving both an average and a sense of how much performance varies across different samples of the same data.

Cross-validation is the standard way to compare models or select hyperparameter tuning settings without touching a final, untouched test set, and it guards against overfitting to one particular split. Its main costs are computation, since the model is trained k times, and, with a small sample size, folds that are too small to give a stable estimate of performance.

Last reviewed September 22, 2026

In the index now

Related terms

Train-test split Dividing a dataset into separate portions for training a model and for evaluating it, so performance is measured on unseen data. Overfitting When a model learns the noise and specific quirks of its training data so closely that it performs poorly on new data. Hyperparameter tuning The process of finding the settings that control how a model learns, such as learning rate or tree depth, rather than what it learns. Sample size The number of observations in a study or test, which determines how precisely and reliably an effect can be measured. Training data The dataset with known outcomes that a machine learning model learns patterns from before being used to predict new cases.

Related tools

JMP Interactive statistical discovery software pairing dynamic data visualization with design of experiments, for engineers and scientists. sktime Open-source Python toolkit that extends scikit-learn conventions to forecasting, classification and clustering of time series.