Install
Storage & Compute terms
Warehouses, lakes, lakehouses and the engines that query them.
Apache Iceberg An open table format adding ACID transactions, schema evolution, and time travel to files stored in a data lake. Apache Parquet An open-source, columnar file format for storing large analytical datasets efficiently in disk or object storage. Columnar storage A file and database layout that stores each column's values together, instead of storing each row together. Data lake A storage repository that holds raw data of any format cheaply, with structure applied only when the data is read. Data lakehouse A storage architecture that adds warehouse-style structure and transactions directly on top of low-cost data lake storage. Data mart A smaller, focused subset of a data warehouse built around the needs of one business function or team. Data virtualization Querying data across multiple, separate systems in place through a single interface, without first copying it. Data warehouse A centralized, structured database optimized for fast historical reporting rather than day-to-day transactions. Feature store A centralized system that stores, versions and serves machine learning features so training and live prediction use identical definitions. High-performance computing (HPC) Using clusters of powerful, tightly linked computers to run computations far larger than a single machine can handle. In-memory database A database that keeps its primary copy of data in RAM instead of on disk, trading persistence for low latency. Massively parallel processing An architecture that splits a query across many independent compute nodes, each processing its own data slice. Materialized view A query result stored physically and refreshed on a schedule or trigger, rather than recomputed on every read. OLAP (online analytical processing) A category of technology optimized for fast, multi-dimensional aggregation and slicing of data for analysis. Open table format A specification that adds transactions, schema management, and versioning to files stored in a data lake. Partitioning Physically splitting a table's data into segments, usually by a column like date, so queries can skip irrelevant ones. Query engine The software component that parses, plans, and executes queries against one or more data sources. Query optimization The process, largely automated by the database, of choosing the fastest execution plan for a given query. Real-time OLAP An analytical database that ingests streaming data continuously and answers aggregation queries with sub-second latency. Schema evolution The ability of a table to accommodate structural changes, like an added or renamed column, without breaking data. Schema-on-read Storing data in its raw form and applying structure only at query time, instead of when it is first loaded. SQL (structured query language) The standard declarative language for defining, querying, and manipulating data in a relational database. Star schema A dimensional modeling pattern with a central fact table connected to descriptive dimension tables, shaped like a star. Time-series database A database purpose-built for data points indexed by time, optimized for fast writes and time-range queries. Vector database A database optimized to store numeric embeddings and quickly find the ones most similar to a given query vector. Zero-copy data sharing Granting another party live access to a dataset in place, without exporting, copying, or moving the files.