In-process & embedded engines · Apache Software Foundation

Apache DataFusion

Open-source, embeddable Rust query engine and toolkit for building custom SQL and DataFrame engines on Apache Arrow.

Visit Apache DataFusion Alternatives

Apache DataFusion is an open-source, extensible query engine written in Rust, built on the Apache Arrow columnar format. Unlike a standalone database, DataFusion is primarily a toolkit: it provides a SQL parser, query planner, and a vectorized, multi-threaded execution engine that other projects embed to build their own analytical databases, DataFrame libraries, or streaming engines without writing an execution engine from scratch. It reads Parquet, CSV, JSON, and Avro natively, supports pluggable table providers for custom data sources, and can be used directly as an in-process SQL/DataFrame library in Rust or Python. Notable projects built on or around DataFusion include InfluxDB IOx and several lakehouse query layers. It runs embedded inside whatever application links it, with no server component of its own.

At a glance

Vendor Apache Software Foundation
Pricing model Open source + paid options
Free tier Yes
Deployment Self-hosted
Open source Yes (Apache-2.0)
Best for Engineers building a custom database, DataFrame library, or streaming engine who don't want to write a query executor from scratch.

Pricing

Free and open source under the Apache 2.0 license; no vendor pricing, as it is a library rather than a hosted product.

Pricing has not been verified yet — see the vendor's site.

Features

  • Embeddable, extensible query engine written in Rust
  • Vectorized, multi-threaded execution on Apache Arrow
  • SQL and DataFrame APIs for Rust and Python
  • Native Parquet, CSV, JSON, and Avro readers
  • Pluggable table providers for custom data sources
  • Cost-based query optimizer and physical planner
  • Used as the execution core inside other databases and tools

Integrations

Profile last reviewed September 21, 2026

Head to head

Alternatives

DuckDB Open-source embedded analytical database that runs SQL queries in-process, without a server, on local or remote files. Polars Open-source, Rust-built in-process DataFrame library offering a fast, multi-threaded alternative to pandas for local analytics. clickhouse-local Single-binary command-line tool that runs ClickHouse's query engine locally over files, without installing a server.

Apache DataFusion in the index now

Terms to know

Data modeling The practice of organizing data into entities, relationships and schemas that make it accurate and efficient to query. OLAP (online analytical processing) A category of technology optimized for fast, multi-dimensional aggregation and slicing of data for analysis. Query engine The software component that parses, plans, and executes queries against one or more data sources.

Related guides

How to choose an in-process analytics engine Pick an embedded analytics engine by workload shape, language runtime and what happens the day you outgrow a single machine.