Glossary

Hyperparameter tuning

The process of finding the settings that control how a model learns, such as learning rate or tree depth, rather than what it learns.

Also called: hyperparameter optimization

Hyperparameters are the settings that control how a model learns, chosen before training begins, as opposed to the parameters, such as a tree's splits or a network's weights, that the model learns from data. Examples include a gradient boosting model's learning rate and number of trees, or a random forest's tree depth. Hyperparameter tuning is the process of searching for the combination of these settings that gives the best performance.

Common search strategies include grid search, exhaustively trying every combination from a defined set of values; random search, sampling combinations at random, which often finds good settings faster than grid search in high-dimensional spaces; and Bayesian optimization, which uses the results of earlier trials to choose more promising settings to try next. Each candidate combination is evaluated using cross-validation rather than the training set alone, so tuning does not simply reward settings that let the model overfit.

Tuning matters because the same algorithm can perform very differently depending on its settings, and poor defaults can leave real accuracy on the table. Its main costs are compute and time, since each trial usually requires retraining the model, which is why AutoML tools increasingly automate the search rather than leaving it to manual trial and error.

Last reviewed September 22, 2026

In the index now

Related terms

Cross-validation A method for estimating how well a model generalizes by repeatedly training and testing it on different splits of the same data. Overfitting When a model learns the noise and specific quirks of its training data so closely that it performs poorly on new data. AutoML (automated machine learning) Tooling that automates parts of building a machine learning model, such as feature selection, algorithm choice and hyperparameter tuning. Gradient boosting An ensemble method that builds trees one at a time, each one correcting the errors made by the trees before it. Random forest An ensemble model that trains many decision trees on random subsets of data and features, then averages their predictions.

Related tools

Comet Hosted experiment-tracking platform that also offers a separate LLM observability product (Opik). Google Vertex AI Google Cloud's unified platform for AutoML, custom model training, and access to foundation models. Kubeflow Open-source, Kubernetes-native toolkit for orchestrating ML pipelines, notebooks, and training jobs. Ray Open-source Python framework for distributing machine-learning training, tuning and general compute across a cluster.

Related guides

How to choose a data science & ML platform The decisions that matter are who builds the models, how automated the modeling should be, and whether you want a cloud-native platform or a portable one.