Collection & Ingestion tools

53 tools filed under Collection & Ingestion, in 6 categories. Open a profile for verified pricing, features and alternatives.

Latest Collection & Ingestion coverage All categories

Change data capture

Tool	Pricing model	Free tier	Open source
Artie Managed real-time data replication that streams row-level database changes to a warehouse with sub-minute latency.	Usage-based	Yes	No
AWS Database Migration Service AWS's managed service for one-time database migration and ongoing log-based change data capture replication.	Usage-based	Yes	No
Debezium Open-source distributed platform for log-based change data capture, streaming database changes into Apache Kafka.	Open source + paid	Yes	Yes
Decodable Fully managed stream-processing platform built on Apache Flink and Debezium for CDC pipelines and real-time transformation.	Usage-based	Yes	No
Maxwell's Daemon Open-source application that reads MySQL binlogs and writes each change as JSON to Kafka, Kinesis or similar streams.	Open source + paid	Yes	Yes
Oracle GoldenGate Oracle's log-based data replication and change data capture technology for real-time database synchronization and migration.	Quote only	No	No
Qlik Replicate Log-based change data capture and replication engine for moving data from enterprise databases and mainframes with minimal source impact.	Quote only	No	No
Streamkap Managed streaming CDC platform that replicates database changes to warehouses and lakes in under a second.	Usage-based	Yes	No
Striim Real-time data integration platform combining log-based CDC with in-flight stream processing, available self-hosted or as managed cloud.	Free tier + paid	Yes	No

Data marketplaces & exchanges

Tool	Pricing model	Free tier	Open source
AWS Data Exchange AWS-native marketplace for finding, subscribing to and billing third-party datasets delivered through S3, Redshift or APIs.	Quote only	Yes	No
Databricks Marketplace Open marketplace inside Databricks for datasets, notebooks and AI models shared via Delta Sharing directly into a workspace.	Quote only	Yes	No
Datarade Platform-agnostic directory and marketplace connecting buyers to thousands of external data providers, not tied to any one cloud.	Quote only	Yes	No
Dawex Data-exchange infrastructure vendor that lets organizations run their own branded internal or external data marketplace.	Quote only	No	No
Google Cloud Datasets Free, Google-hosted public datasets (census, geospatial, health, finance and more) queryable directly in BigQuery.	Free	Yes	No
Nasdaq Data Link Marketplace and API for financial, economic and alternative datasets, formerly known as Quandl, now owned by Nasdaq.	Quote only	Yes	No
Snowflake Marketplace In-platform listing of third-party datasets, apps and AI products that can be queried directly inside a Snowflake account.	Quote only	Yes	No

ELT & data integration

Tool	Pricing model	Free tier	Open source
Airbyte Open-source data movement platform with 600+ connectors, run self-hosted for free or as managed Airbyte Cloud.	Usage-based	Yes	Yes
AWS Glue Serverless AWS ETL service using Apache Spark under the hood, billed per Data Processing Unit-hour with no infrastructure to manage.	Usage-based	Yes	No
Azure Data Factory Microsoft's cloud ETL/ELT and orchestration service for building and scheduling data pipelines across Azure and hybrid sources.	Usage-based	Yes	No
Estuary Flow Real-time data integration platform that unifies batch ELT and streaming CDC pipelines with sub-second latency.	Usage-based	Yes	No
Fivetran Managed ELT platform with 700+ prebuilt connectors that automatically syncs source data into a cloud warehouse or lake.	Usage-based	Yes	No
Google Cloud Dataflow Fully managed, serverless service for running unified batch and streaming data pipelines built on Apache Beam.	Usage-based	No	No
Hevo Data No-code, fully managed ELT platform billed by monthly event volume, with 150+ connectors and near real-time sync.	Usage-based	Yes	No
Informatica Enterprise-grade cloud data integration and management suite, priced by consumption of Informatica Processing Units (IPUs).	Quote only	No	No
Integrate.io Flat-fee ELT and ETL platform with unlimited data volume, aimed at teams that want predictable pricing over metered billing.	Subscription	No	No
Matillion Cloud-native ELT platform with a low-code visual pipeline designer that pushes transformations down into the target warehouse.	Quote only	No	No
Meltano Open-source, code-first ELT framework built on the Singer specification, with an optional managed Meltano Cloud.	Quote only	Yes	Yes
Portable Managed ELT platform specializing in long-tail, hard-to-find connectors, priced as a flat monthly fee per enabled data flow.	Subscription	No	No
Rivery Cloud ELT platform combining ingestion, orchestration and reverse ETL, billed by consumption of River Data Units (RDU/BDU credits).	Usage-based	No	No
Stitch Simple, developer-friendly managed ELT service billed by monthly row volume, built on the open-source Singer connector framework.	Usage-based	No	No
Talend Enterprise data integration and data quality suite, now sold as Qlik Talend Cloud after Qlik's acquisition of Talend.	Quote only	No	No

Event collection & tracking pipelines

Tool	Pricing model	Free tier	Open source
Freshpaint Compliance-focused event tracking pipeline that auto-captures behavioral data while filtering PHI/PII before it reaches destinations.	Quote only	No	No
Jitsu Open-source event collection platform positioned as a self-hostable, developer-friendly alternative to Segment.	Free tier + paid	Yes	Yes
Snowplow Behavioral data pipeline that creates a fully owned, schema-validated event stream in the customer's own warehouse or lake.	Quote only	Yes	No
Twilio Segment Customer data pipeline that collects events once via SDKs and routes them to 700+ downstream tools and warehouses.	Quote only	No	No

Event streaming

Tool	Pricing model	Free tier	Open source
Amazon Kinesis AWS's managed service for real-time data streaming, ingesting and processing continuous data at scale without managing servers.	Usage-based	No	No
Apache Kafka Open-source distributed event streaming platform for high-throughput publish-subscribe messaging and durable event logs.	Open source + paid	Yes	Yes
Apache Pulsar Open-source, cloud-native distributed messaging and streaming platform with built-in multi-tenancy and geo-replication.	Open source + paid	Yes	Yes
Azure Event Hubs Microsoft's managed big-data streaming service, exposing a Kafka-compatible endpoint for event ingestion on Azure.	Usage-based	No	No
Confluent Fully managed Kafka service from Kafka's original creators, adding schema registry, connectors and stream processing on top.	Usage-based	Yes	No
Google Cloud Pub/Sub Google Cloud's fully managed, serverless messaging service for asynchronous event ingestion between applications and analytics pipelines.	Usage-based	Yes	No
Redpanda Kafka-API-compatible streaming platform built in C++ without ZooKeeper or the JVM, available self-managed or as managed cloud.	Usage-based	Yes	No

Web scraping & data extraction

Tool	Pricing model	Free tier	Open source
Apify Hosted crawler platform where scrapers run as cloud 'Actors', with a marketplace of pre-built ones and a proxy add-on.	Subscription	Yes	No
Bright Data Proxy network and web-data platform for large-scale scraping, with residential/datacenter IPs and hosted scraper APIs.	Usage-based	Yes	No
Diffbot Extraction API that uses machine-learning models to return structured data (articles, products, discussions) from any page without site-specific rules.	Subscription	Yes	No
Firecrawl Hosted fetch API that turns any URL or site crawl into clean Markdown/JSON, built for feeding LLM and RAG pipelines.	Free tier + paid	Yes	Yes
Import.io Managed web-data-as-a-service platform combining visual extraction setup with a team that maintains scrapers against defended sites.	Subscription	Yes	No
Octoparse No-code visual scraper: point-and-click a desktop or cloud workflow to extract site data without writing code.	Free tier + paid	Yes	No
Oxylabs Proxy network provider (residential, datacenter, ISP, mobile) with add-on scraper APIs for unblocking and structured extraction.	Usage-based	Yes	No
Playwright Open-source browser-automation library (not a scraping product) that many teams use directly to script and scrape JS-heavy sites.	Open source + paid	Yes	Yes
ScrapingBee Simple fetch API that renders JavaScript and rotates proxies, returning page HTML or extracted data for a credit per request.	Subscription	No	No
Scrapy Open-source Python framework you run yourself to write, schedule and scale web crawlers and spiders.	Open source + paid	Yes	Yes
Zyte Hosted crawler platform and single unblocking API, built by the maintainers of the Scrapy framework.	Usage-based	Yes	No