How to choose a lakehouse platform

"Lakehouse" describes an architecture, not a product: data stays in cheap object storage as files, but a metadata layer adds the transactional guarantees and structure that used to require loading everything into a warehouse first. Shopping this category actually means answering two separate questions that vendors like to blur together. First, which open table format will your data be stored in — Apache Iceberg, Delta Lake, Apache Hudi or Apache Paimon? Second, what manages and queries those tables — a full platform such as Databricks, Cloudera Data Platform or Dremio, or your own Spark, Trino or Flink jobs pointed at the raw files? Getting the two decisions tangled is how teams end up locked into a platform for a format choice they never meant to make permanently.

Teams that don't need this: if your data comfortably fits in a conventional warehouse and nobody outside a small analytics team touches it, the operational complexity of a lakehouse buys you optionality you may not need yet.

Decision 1: which table format

All four formats solve the same underlying problem — turning a directory of Parquet or ORC files into something with CDC-safe transactions, schema evolution and time travel — but they optimize for different access patterns.

• Apache Iceberg is engine-agnostic by design and governed independently by the Apache Software Foundation, which is why it has become a common interchange format across vendors including Snowflake, Databricks, Dremio and Starburst. It is the safest default if you don't know yet which engines will need to read the data.
• Delta Lake originated at Databricks and is now under the Linux Foundation, but its deepest, most native integration is still with Spark and Databricks specifically. Choose it if your compute is already Spark-centric.
• Apache Hudi was built at Uber specifically for high-volume, record-level upserts and deletes — the pattern behind CDC ingestion and regulatory deletion requirements. It ships built-in indexing and incremental queries that the others don't emphasize as strongly.
• Apache Paimon comes out of the Apache Flink community and uses an LSM-tree storage layout to keep continuous streaming writes fast while still supporting efficient batch reads on the same table — the format to look at when the write side is a live stream, not a periodic batch load.

Decision 2: managed platform or assemble it yourself

A table format alone is just files and metadata; something still has to ingest into it, run compaction, manage a catalog, and answer queries. Databricks, Cloudera Data Platform, Dremio and Onehouse sell that "something" as a managed product. The alternative — pointing your own Spark, Trino or Flink deployment at Iceberg, Delta or Hudi tables directly — costs more engineering time but avoids paying a platform margin and keeps you free to swap the compute layer later.

Who each platform is actually for

Databricks is built by Apache Spark's original creators and is the natural choice if your data engineering, SQL analytics and machine learning work is already Spark-centric and you want one platform spanning all three, with Unity Catalog for governance and Delta Lake as the native format. It bills in Databricks Units per second on top of your own cloud infrastructure cost.

Cloudera Data Platform targets large, often regulated enterprises with existing Hadoop-ecosystem investment (HDFS, Hive, Kafka, Spark) that need a governed lakehouse without a full platform replacement, and — unlike the cloud-only options here — genuinely supports hybrid and on-premises deployment.

Dremio takes a different angle: rather than managing ingestion and storage, it is a SQL query and semantic-layer engine built on Apache Iceberg and Apache Arrow that lets analysts and BI tools query lake data directly, with reflections (auto-maintained materializations) standing in for the warehouse copy you'd otherwise build. It suits teams that want governed self-service SQL over the lake without moving data into a separate warehouse.

Onehouse, founded by Apache Hudi's original creators, is a managed ingestion and table-optimization service built around a universal metadata layer that can expose the same underlying data as Hudi, Iceberg or Delta tables — useful specifically when different downstream tools in your organization are already committed to different formats.

Open source doesn't mean free of operational cost

Every table format here is free to use, but running compaction, clustering, file cleanup and a catalog service at production scale is real, ongoing engineering work — the kind Databricks, Cloudera, Dremio and Onehouse are selling relief from. Before deciding to self-manage a format on your own Spark or Flink cluster, be honest about who on the team will own that operational surface, and whether the platform's usage-based fee is actually cheaper than the engineering time it replaces.

Deployment and compliance

Databricks and Onehouse are cloud-only. Cloudera is the option built for hybrid and on-premises deployment, which matters if data residency or air-gapped requirements rule out a pure cloud service. Dremio offers both a managed cloud and self-hosted software edition. If you assemble your own stack on open table formats, deployment is simply wherever your Spark, Trino or Flink cluster runs.

How pricing scales

Databricks and Dremio are usage-based, metered by compute unit (DBUs and DCUs respectively) on top of underlying cloud infrastructure cost. Cloudera and Onehouse are quote-only, typical of enterprise and consumption-based licensing that doesn't publish a rate card. The table formats themselves — Iceberg, Delta Lake, Hudi, Paimon — carry no license fee; your cost is entirely the compute that reads and writes them.

A shortlist by situation

• Already Spark-centric and want engineering, ML and SQL under one platform: Databricks.
• Large regulated enterprise with existing Hadoop investment needing hybrid or on-premises deployment: Cloudera Data Platform.
• Want governed SQL directly over lake data without a separate warehouse copy: Dremio.
• Heavy upsert/delete or CDC ingestion and want it managed without running Hudi clusters yourself: Onehouse.
• Assembling your own stack and just need the table format: Apache Iceberg for the broadest engine support, Delta Lake for Spark/Databricks-native integration, Apache Hudi for upsert-heavy CDC pipelines, Apache Paimon for streaming-first writes via Flink.

Questions to ask vendors or in a trial

1. Which table format(s) does it read and write natively, versus only through a connector?
2. Who runs compaction, clustering and catalog services, and what does that cost at our data volume?
3. Can other engines already in our stack read the same tables without re-exporting?
4. If we leave this platform in two years, do we keep our data in a genuinely open format, or does it require conversion?

Common mistakes

Picking a full platform before checking whether an open table format on existing compute already covers the need is the most expensive one to reverse. A close second is assuming "open table format" means permanent vendor independence — a platform's specific catalog, extensions or default settings can still create real switching friction even on top of Iceberg or Delta. The third is underestimating the ongoing cost of self-managed compaction and clustering, which is invisible until the tables are large enough for it to matter.

Where to go next

Apache Iceberg vs Delta Lake and Apache Hudi vs Apache Iceberg cover the two most-asked table format questions in more depth. See every tool in this category for the full list.