Feature engineering

Feature engineering is the process of creating, transforming or selecting the input variables, called features, that a model is trained on. Raw data, such as a timestamp or a free-text field, is rarely in the most useful form for a model; feature engineering turns it into variables that make the target relationship easier to learn, for example converting a timestamp into "day of week" and "hours since last purchase."

Typical techniques include combining existing fields into ratios or differences, encoding categorical variables numerically, aggregating transaction-level data up to a customer level, and applying dimensionality reduction to compress a large number of correlated features. It sits downstream of data modeling and upstream of model training, and is closely tied to domain knowledge: a practitioner who understands the business problem can often create features no automated process would find.

Well-designed features often improve model accuracy more than switching algorithms does, which is why feature engineering is considered one of the highest-leverage parts of a predictive project. Common pitfalls include leaking information from the target into a feature, for example using a value only known after the outcome occurred, and creating so many correlated features that the model overfits the training data. Reusable features are increasingly managed centrally in a feature store.