Technology terms

The analytics stack, layer by layer: ingestion, storage, modeling, governance, BI, ML platforms, AI tools, activation and operations.

Apache Kafka An open-source distributed event streaming platform that stores records in ordered, durable, replayable logs called topics. API integration Connecting two systems by having one call the other's application programming interface to exchange data. Change data capture (CDC) A technique for detecting and streaming only the rows that changed in a source system, instead of reloading everything. Connector A prebuilt, reusable component that handles the specifics of extracting data from or loading data into a system. Consent mode A Google framework that adjusts how its tags behave based on a visitor's cookie consent choices. Data contract A formal, versioned agreement on a dataset's schema and semantics between the team producing it and its consumers. Data deduplication Identifying and removing records that represent the same real-world entity or event more than once. Data ingestion Moving data from its source systems into a storage or processing system where it can be transformed and analyzed. Data layer A structured object on a page that exposes site and event data for a tag management system to read. Data pipeline The automated sequence of steps that moves data from source systems to a destination, transforming it along the way. ELT (extract, load, transform) A data integration pattern that loads raw data first and transforms it afterward using the warehouse's own compute. ETL (extract, transform, load) A data integration pattern that transforms data before loading it, unlike ELT which transforms after loading. Event tracking Capturing specific user actions, such as clicks or purchases, as discrete, named pieces of data. First-party data Data a company collects directly from its own customers and properties, rather than acquiring it from outside sources. Message broker Middleware that routes messages between producing and consuming systems so they don't need to connect to each other directly. Server-side tagging Sending tracking events through a first-party server rather than firing vendor tags directly from the browser. Tag management A system for adding and configuring marketing and analytics tracking tags without editing a site's code. UTM parameters Tags added to a URL that record which source, medium, and campaign sent a visitor to a site. Webhook An automated HTTP callback a system sends to a URL the moment a specific event happens, instead of waiting to be asked. Apache Iceberg An open table format adding ACID transactions, schema evolution, and time travel to files stored in a data lake. Apache Parquet An open-source, columnar file format for storing large analytical datasets efficiently in disk or object storage. Columnar storage A file and database layout that stores each column's values together, instead of storing each row together. Data lake A storage repository that holds raw data of any format cheaply, with structure applied only when the data is read. Data lakehouse A storage architecture that adds warehouse-style structure and transactions directly on top of low-cost data lake storage. Data mart A smaller, focused subset of a data warehouse built around the needs of one business function or team. Data virtualization Querying data across multiple, separate systems in place through a single interface, without first copying it. Data warehouse A centralized, structured database optimized for fast historical reporting rather than day-to-day transactions. Feature store A centralized system that stores, versions and serves machine learning features so training and live prediction use identical definitions. High-performance computing (HPC) Using clusters of powerful, tightly linked computers to run computations far larger than a single machine can handle. In-memory database A database that keeps its primary copy of data in RAM instead of on disk, trading persistence for low latency. Massively parallel processing An architecture that splits a query across many independent compute nodes, each processing its own data slice. Materialized view A query result stored physically and refreshed on a schedule or trigger, rather than recomputed on every read. OLAP (online analytical processing) A category of technology optimized for fast, multi-dimensional aggregation and slicing of data for analysis. Open table format A specification that adds transactions, schema management, and versioning to files stored in a data lake. Partitioning Physically splitting a table's data into segments, usually by a column like date, so queries can skip irrelevant ones. Query engine The software component that parses, plans, and executes queries against one or more data sources. Query optimization The process, largely automated by the database, of choosing the fastest execution plan for a given query. Real-time OLAP An analytical database that ingests streaming data continuously and answers aggregation queries with sub-second latency. Schema evolution The ability of a table to accommodate structural changes, like an added or renamed column, without breaking data. Schema-on-read Storing data in its raw form and applying structure only at query time, instead of when it is first loaded. SQL (structured query language) The standard declarative language for defining, querying, and manipulating data in a relational database. Star schema A dimensional modeling pattern with a central fact table connected to descriptive dimension tables, shaped like a star. Time-series database A database purpose-built for data points indexed by time, optimized for fast writes and time-range queries. Vector database A database optimized to store numeric embeddings and quickly find the ones most similar to a given query vector. Zero-copy data sharing Granting another party live access to a dataset in place, without exporting, copying, or moving the files. Common table expression A named, temporary result set defined at the start of a SQL query and referenced later in that query. Data modeling The practice of organizing data into entities, relationships and schemas that make it accurate and efficient to query. Data vault A modeling methodology that splits data into hubs, links, and satellites to make a warehouse auditable and adaptable to change. dbt (data build tool) An open-source tool from dbt Labs that lets analytics teams write data transformations as version-controlled, tested SQL. Denormalization Deliberately introducing redundancy into a data model to reduce joins and speed up read queries. Dimension table A table of descriptive attributes, such as customer or product details, that a fact table references for context. Fact table The table in a dimensional model that stores measurable, numeric events, each row linked to the dimensions describing it. Incremental model A transformation that updates only new or changed rows on each run instead of rebuilding the entire table. Normalization Structuring a database so each fact is stored in exactly one place, reducing redundancy and inconsistency. One big table A denormalized modeling approach that joins facts and dimensions into a single wide table instead of many. Reverse ETL The practice of syncing modeled data from a warehouse back out into operational tools like a CRM or ad platform. Semantic layer A layer that defines business metrics and dimensions once, centrally, so every tool and report uses the same numbers. Slowly changing dimension The techniques for recording how a dimension table's attributes change over time without breaking historical reporting. Snowflake schema A dimensional model where dimension tables are normalized into multiple related tables instead of one flat table. Surrogate key An artificial, meaningless identifier generated by the warehouse to uniquely identify a dimension or fact row. Window function A SQL calculation performed across a set of related rows without collapsing them into a single output row. Business glossary A controlled list of business terms and their agreed definitions, kept separate from the technical fields that implement them. Data catalog A searchable inventory of an organization's data assets, with descriptions, ownership and usage attached. Data classification Categorizing data by sensitivity or type so that handling, access, and protection rules can be applied consistently and automatically. Data dictionary A structured reference listing every table, column, and field in a database along with its type, meaning, and constraints. Data drift A change in the statistical properties of input data over time that can silently degrade a deployed model's accuracy. Data governance The policies, roles and processes that determine how data is defined, owned, accessed and kept compliant. Data lineage A traceable record of where data came from, how it was transformed, and where it is used downstream. Data masking Replacing sensitive values with realistic but fake substitutes so data can be used for testing or analysis without exposure. Data observability Continuous automated monitoring of pipelines and datasets to catch freshness, volume and quality problems early. Data profiling Automatically scanning a dataset to summarize its structure, value distributions, and quality before it is trusted or modeled. Data provenance The documented history of where a dataset came from and every transformation applied to it before its current state. Data quality The degree to which data is accurate, complete, consistent and fit for the purpose it is being used for. Data retention policy A documented rule for how long different categories of data are kept before they are deleted or archived. Data standards Agreed formats, fields, and definitions that let datasets from different agencies or systems be combined reliably. Data stewardship The assignment of named people who are accountable for the quality, definition, and appropriate use of specific data domains. Data validation Checking that data meets explicit, predefined rules for type, range, format, or consistency before it is used or trusted. Master data management (MDM) The discipline of maintaining one authoritative, consistent record for core entities like customers or products. Metadata management The discipline of capturing, organizing, and maintaining data about data so it can be found, trusted, and used consistently. Model monitoring Ongoing tracking of a deployed model's inputs, predictions and outcomes to catch accuracy loss, drift or failures after launch. Pseudonymization Replacing identifying data with a reversible token, keeping the mapping back to real identities separate and restricted. Role-based access control (RBAC) Granting data or system permissions based on a user's assigned role rather than configuring access for each person individually. Row-level security (RLS) Restricting which rows of a table a user can query, rather than granting or denying access to the whole table. Business intelligence (BI) The practice of turning company data into dashboards, reports and metrics that guide business decisions. Dashboard A visual display of the metrics a team needs to monitor, usually updated on a schedule or in real time. Data storytelling Combining data, visuals and narrative so an audience understands an analysis and knows what to do about it. Data visualization Representing data graphically so that patterns, trends and outliers are easier to see than in a table of numbers. Embedded analytics Charts, reports or dashboards built directly into another application instead of a separate BI tool. KPI (key performance indicator) A measurable value tracked over time to show whether a team or process is on track toward a goal. Process mining Reconstructing how a business process actually runs from system event logs rather than from how it is documented. Self-service BI BI tooling that lets business users build their own reports and explore data without waiting on IT or analytics teams. Semantic search Search that matches results to a query's meaning rather than its exact keywords, using vector similarity between embeddings. What-if analysis Testing how a model's output changes when one or more input assumptions are changed, to explore specific alternative outcomes. AutoML (automated machine learning) Tooling that automates parts of building a machine learning model, such as feature selection, algorithm choice and hyperparameter tuning. Classification A supervised learning task that assigns each input to one of a fixed set of discrete categories or labels. Clustering An unsupervised technique that groups data points so items within a group are more similar to each other than to items outside it. Concept drift A change over time in the underlying relationship between a model's inputs and the outcome it predicts. Confusion matrix A table comparing a classifier's predicted labels against actual labels, breaking results into true and false positives and negatives. Decision tree A supervised model that predicts an outcome by following a sequence of if-then splits on feature values, shaped like a tree. Deep learning Machine learning using neural networks with many layers, able to learn complex patterns directly from raw data such as images or text. Dimensionality reduction Techniques that compress a dataset with many variables into fewer dimensions while preserving as much meaningful information as possible. Explainable AI (XAI) Techniques and models that make a machine learning system's predictions understandable to humans, rather than a black box. F1 score The harmonic mean of precision and recall, giving a single score that balances both when neither error type is clearly more costly. Feature engineering Creating, transforming or selecting the input variables a model is trained on, to make the underlying pattern easier for it to learn. Fine-tuning Further training a pretrained model on a smaller, targeted dataset to adapt its behavior to a specific task or domain. Gradient boosting An ensemble method that builds trees one at a time, each one correcting the errors made by the trees before it. Hyperparameter tuning The process of finding the settings that control how a model learns, such as learning rate or tree depth, rather than what it learns. K-means clustering A clustering algorithm that partitions data into k groups by repeatedly assigning points to the nearest of k cluster centers. Knowledge graph A network of entities and the labeled relationships between them, used to represent and query structured knowledge. Machine learning Building models that learn patterns from data to make predictions or decisions, rather than following hand-coded rules. MLOps (machine learning operations) The set of practices for deploying, monitoring and maintaining machine learning models reliably in production, extending DevOps to ML systems. Neural network A model made of layers of connected nodes that learns to map inputs to outputs by adjusting connection weights during training. Overfitting When a model learns the noise and specific quirks of its training data so closely that it performs poorly on new data. Precision and recall Two complementary classifier metrics; precision is how many predicted positives were correct, recall is how many actual positives were found. Random forest An ensemble model that trains many decision trees on random subsets of data and features, then averages their predictions. Reinforcement learning A learning approach where an agent learns a strategy by taking actions in an environment and receiving rewards or penalties over time. Retrieval-augmented generation (RAG) An architecture that retrieves relevant documents at query time and feeds them to a language model so its answers are grounded in sources. ROC AUC (area under the ROC curve) A single number summarizing how well a classifier separates positive from negative cases across every possible decision threshold. SHAP values (SHapley Additive exPlanations) A method for explaining an individual prediction by fairly attributing it to the contribution of each input feature. Supervised learning A machine learning approach where a model learns from examples that already include the correct answer, to predict that answer on new data. Train-test split Dividing a dataset into separate portions for training a model and for evaluating it, so performance is measured on unseen data. Training data The dataset with known outcomes that a machine learning model learns patterns from before being used to predict new cases. Unsupervised learning A machine learning approach that finds structure or patterns in data that has no labeled correct answer to learn from. AI copilot An AI tool embedded in a work application that assists a human with suggestions or drafts, rather than working unsupervised. Augmented analytics Using AI and machine learning to automate parts of data preparation, insight discovery, and explanation in analytics tools. Conversation intelligence Software that records, transcribes and analyzes sales or support calls to surface talk patterns, objections and coaching cues. Large language model (LLM) A neural network trained on vast amounts of text to predict and generate language, the technology behind most modern AI assistants. Natural language query Asking a data question in plain language and getting an answer without writing code or a query language. Text-to-SQL Technology that translates a plain-language question directly into a runnable SQL query against a database. Composable CDP A customer data platform assembled from separate best-of-breed tools on top of an organization's own warehouse, rather than one bundled product. Consent management platform (CMP) Software that collects, records, and enforces a person's consent choices for cookies and data collection across a site or app. Customer 360 A single, unified profile of a customer that consolidates data from every touchpoint and system into one view. Customer data platform (CDP) Software that unifies customer data from multiple sources into persistent profiles usable by marketing teams. Data clean room A controlled environment where two or more parties analyze combined data together without either side seeing the other's raw records. Identity resolution Matching data from different sources and devices to a single underlying person or household despite no shared identifier. Backfill Reprocessing historical data through a pipeline, typically after a bug fix, schema change or new metric definition. Batch processing Processing data in large, scheduled groups rather than continuously as each record arrives. Data fabric An architecture that uses metadata and automation to integrate and provide unified access to data across many sources. Data freshness How current a dataset is relative to real time, measured as the lag between an event happening and it being queryable. Data mesh A decentralized approach where domain teams own and publish their own data as products, instead of one central team. Data product A dataset or data service built, owned and maintained with product-level rigor for a defined set of consumers. Data SLA (service-level agreement) A formal commitment about a dataset's freshness, availability or quality, with defined targets and consequences. DataOps Applying DevOps practices, such as automated testing, version control and continuous delivery, to data pipelines. Directed acyclic graph (DAG) A graph of one-way dependency edges between tasks, with no cycles, used to define the run order of a pipeline. Event-driven architecture (EDA) A system design where components communicate by producing and reacting to events, rather than calling each other directly. Idempotency The property where running an operation multiple times leaves the same result as running it once. Micro-batching Processing data in very small, frequent batches to approximate streaming without a dedicated stream processor. Workflow orchestration Scheduling, sequencing and monitoring interdependent data tasks so each step runs only after the ones it depends on succeed. Clinical decision support (CDS) Software that gives clinicians patient-specific alerts or recommendations at the point of care. Electronic health record (EHR) The digital, longitudinal record of a patient's health data, maintained across their encounters with a provider. Grid analytics Analyzing sensor and meter data from the electricity grid to balance load, detect faults, and plan investment.