Glossary

Data ingestion

Moving data from its source systems into a storage or processing system where it can be transformed and analyzed.

Data ingestion is the process of bringing data from wherever it originates, an application database, a SaaS tool, a stream of sensor readings, into a system, typically a data lake or warehouse, where it can be stored, transformed and analyzed. It is the first stage of most data pipelines, before any transformation happens.

Ingestion can happen in batches, pulling a full or incremental extract on a schedule, or continuously via streaming and event pipelines, depending on how quickly the data is needed downstream. It is commonly built using pre-built connectors for common sources, or custom API integrations and webhooks for systems without one. Ingestion is distinct from the transformation step in ETL/ELT: ingestion is concerned with getting data in reliably, transformation with reshaping it once it has landed.

Ingestion matters because it is where data quality problems most often originate or first become visible, missing records, schema drift from a source system, duplicate loads from a retried job, and catching issues here is far cheaper than catching them after they've propagated through several downstream models. Common pitfalls include ingesting a full table on every run when an incremental approach would be far cheaper, and not handling schema changes in the source gracefully, which can silently drop or corrupt fields.

Last reviewed September 22, 2026

In the index now

Related terms

Connector A prebuilt, reusable component that handles the specifics of extracting data from or loading data into a system. API integration Connecting two systems by having one call the other's application programming interface to exchange data. Webhook An automated HTTP callback a system sends to a URL the moment a specific event happens, instead of waiting to be asked. ETL (extract, transform, load) A data integration pattern that transforms data before loading it, unlike ELT which transforms after loading. Data pipeline The automated sequence of steps that moves data from source systems to a destination, transforming it along the way.

Related tools

Adobe Customer Journey Analytics Enterprise analytics that stitches web, app, call-centre and offline data into cross-channel customer journeys. Adobe Experience Platform Enterprise data platform underlying Adobe's Real-Time CDP, unifying data across the Adobe Experience Cloud suite. Arize AI Hosted AI observability platform that monitors deployed ML models and LLM applications for drift, quality, and performance issues. Babel Street Multilingual OSINT and identity-resolution platform used by defense, intelligence and law-enforcement analysts. Imply Managed and self-hosted Apache Druid platform built by Druid's original creators. Arize Phoenix Open-source, local-first tracing and evaluation library for LLM and agent applications, with an optional path to Arize's commercial platform.

Related guides

How to choose a time-series database Pick a time-series database by whether you need relational joins, what protocol your data already speaks, and how much cardinality you actually have. How to choose a web scraping tool Pick a web scraping tool by how much of the pipeline you want to own — proxies, rendering, extraction — not by whichever demo pulled a product page fastest.