License & Deployment Mix: 39 tools – 27 OSS, 12 SaaS. (OSS and SaaS counts can overlap when an open-source tool also offers a vendor-hosted edition.)
What Is a Data Warehouse?
A data warehouse is a centralized repository optimized for analytical queries across large volumes of structured data. Unlike transactional databases (OLTP) that handle fast reads and writes of individual records, data warehouses use columnar storage, compression, and pre-aggregation to deliver fast analytical performance over billions of rows.
Traditional data warehouses (Snowflake, BigQuery, Redshift) store structured, cleaned data loaded through ETL/ELT pipelines. They enforce schemas, optimize for aggregation queries, and serve as the single source of truth for business intelligence and reporting.
The information on these pages was researched by a combination of human review and large language models. To suggest an addition or correction, please contact us. Prepared by Rhodium Systems Inc., author of the ResorsIT platform — a unified IT operations management platform for IT teams and MSPs that integrates a curated suite of open-source, commercial, and SaaS applications into a single system with shared identity, single sign-on, access control, and a common audit trail. Use this catalogue only as a starting point for your own research, and review any tool carefully against your own requirements before relying on it. Catalogue data version 2026.197.
Comparison
This comparison covers the OLAP, ETL, lakehouse, and data catalog landscape.
SSO / OIDC Comparison
OIDC is the preferred SSO protocol; the notes below assume Authentik as the external identity provider.
| Tool | OIDC | SAML | SCIM | Authentik Notes |
|---|---|---|---|---|
| ClickHouse | No | No | No | HTTP Basic / LDAP; reverse proxy for OIDC |
| Apache Druid | Yes | No | No | OIDC via Pac4j; Authentik as IdP |
| StarRocks | No | No | No | LDAP auth; reverse proxy for OIDC |
| Apache Doris | No | No | No | LDAP auth; reverse proxy for OIDC |
| Apache Pinot | No | No | No | Basic auth only; reverse proxy for OIDC |
| Greenplum | No | No | No | LDAP/Kerberos; reverse proxy for OIDC |
| MonetDB | No | No | No | Native auth only; reverse proxy for OIDC |
| Snowflake | Yes | Yes | Yes | Native OIDC, SAML, SCIM; Authentik as IdP |
| BigQuery | Yes* | Yes | Yes | Google IAM; Authentik via Workforce Identity |
| Redshift | Yes* | Yes | No | AWS IAM; Authentik via SAML federation |
| Azure Synapse | Yes* | Yes | Yes | Entra ID; Authentik federation possible |
| Databricks | Yes | Yes | Yes | Native OIDC and SAML; SCIM provisioning |
| MotherDuck | Yes | No | No | OIDC via Google/GitHub; custom OIDC possible |
| Trino | Yes | No | No | Native OIDC support; Authentik as IdP |
| DuckDB | N/A | N/A | N/A | Embedded engine; no built-in auth |
| Presto | Yes | No | No | OIDC via pluggable auth; Authentik as IdP |
| Dremio | Yes | Yes | No | Native OIDC and SAML; Authentik as IdP |
| Spark SQL | No | No | No | Kerberos/LDAP; reverse proxy for OIDC |
| Starburst | Yes | Yes | Yes | Native OIDC, SAML, SCIM; Authentik as IdP |
| DataFusion | N/A | N/A | N/A | Library; no built-in auth |
| Apache Iceberg | N/A | N/A | N/A | Format spec; auth via catalog (REST/Hive) |
| Delta Lake | N/A | N/A | N/A | Format spec; auth via engine |
| Apache Hudi | N/A | N/A | N/A | Format spec; auth via engine |
| DuckLake | N/A | N/A | N/A | Format spec; auth via catalog DB |
| Apache Paimon | N/A | N/A | N/A | Format spec; auth via engine |
| dbt | N/A | N/A | N/A | CLI tool; auth via target warehouse |
| Apache Airflow | Yes | No | No | Native OIDC via Flask-AppBuilder; Authentik as IdP |
| Dagster | Yes | Yes | No | OIDC and SAML in Dagster Cloud; OSS via reverse proxy |
| Prefect | Yes | Yes | No | OIDC and SAML in Prefect Cloud; OSS via reverse proxy |
| Meltano | N/A | N/A | N/A | CLI tool; no built-in web UI auth |
| Apache NiFi | Yes | No | No | Native OIDC support; Authentik as IdP |
| Singer | N/A | N/A | N/A | CLI specification; no built-in auth |
| Airbyte | Yes | No | No | OIDC in Cloud/Enterprise; OSS Keycloak-based |
| Fivetran | Yes | Yes | Yes | Native OIDC, SAML, SCIM; Authentik as IdP |
| Stitch | No | Yes | No | SAML SSO; Authentik as SAML IdP |
| Matillion | Yes | Yes | No | Native OIDC and SAML; Authentik as IdP |
| DataHub | Yes | No | No | Native OIDC; Authentik as IdP |
| OpenMetadata | Yes | Yes | No | Native OIDC and SAML; Authentik as IdP |
* = via cloud provider identity federation
Best SSO support: Snowflake, Databricks, Starburst, Fivetran (OIDC + SAML + SCIM)
No built-in auth: DuckDB, DataFusion, dbt, Meltano, Singer (CLI or library tools)
Format specs: Iceberg, Delta Lake, Hudi, DuckLake, Paimon inherit auth from the engine or catalog that accesses them
API Comparison
| Tool | API Type | Auth | Webhooks | SDKs |
|---|---|---|---|---|
| ClickHouse | HTTP, Native TCP | Basic, JWT | No | Python, Go, Java, JS, Rust |
| Apache Druid | REST | Basic, OIDC | No | Python, Java |
| StarRocks | MySQL protocol | MySQL auth | No | JDBC, Python |
| Apache Doris | MySQL, HTTP | MySQL auth, token | No | JDBC, Python |
| Apache Pinot | REST | Basic, token | No | Java, Python |
| Greenplum | PostgreSQL protocol | PG auth | No | libpq, JDBC, Python |
| MonetDB | MAPI protocol | Native auth | No | Python, Java, ODBC |
| Snowflake | REST, SQL | OAuth2, key pair | Yes | Python, Go, Java, JS, .NET |
| BigQuery | REST, gRPC | OAuth2 | Yes | Python, Go, Java, JS, .NET |
| Redshift | PostgreSQL protocol | IAM, PG auth | No | JDBC, ODBC, Python |
| Azure Synapse | REST, T-SQL | OAuth2, AAD | Yes | .NET, Python, Java |
| Databricks | REST, SQL | OAuth2, PAT | Yes | Python, Java, Go, JS, R |
| MotherDuck | DuckDB protocol | Token | No | Python, JS |
| Trino | REST, JDBC | OIDC, JWT, Basic | No | JDBC, Python, Go |
| DuckDB | In-process | N/A | N/A | Python, R, Java, JS, Rust, Go |
| Presto | REST, JDBC | OIDC, Basic | No | JDBC, Python |
| Dremio | REST, JDBC, ODBC | OAuth2, PAT | No | JDBC, ODBC, Python |
| Spark SQL | Thrift, REST | Kerberos, token | No | Python, Scala, Java, R |
| Starburst | REST, JDBC | OIDC, JWT | No | JDBC, Python |
| DataFusion | Library API | N/A | N/A | Rust, Python |
| dbt | CLI, Cloud API | Token (Cloud) | Yes (Cloud) | Python |
| Apache Airflow | REST | JWT, session | Yes | Python |
| Dagster | GraphQL, REST | Token | Yes (Cloud) | Python |
| Prefect | REST | Token | Yes (Cloud) | Python |
| Meltano | CLI | N/A | No | Python |
| Apache NiFi | REST | Bearer, cert | No | Python, Java |
| Singer | CLI (pipes) | N/A | No | Python |
| Airbyte | REST | Bearer | Yes | Python, Java |
| Fivetran | REST | API key | Yes | Python, Go |
| Stitch | REST | API token | No | Python |
| Matillion | REST | API key | Yes | None official |
| DataHub | REST, GraphQL | Token | No | Python, Java |
| OpenMetadata | REST | JWT, Basic | Yes | Python, Java |
DW Feature Comparison
Self-Hosted OLAP
| Feature | ClickHouse | Druid | StarRocks | Doris | Pinot | Greenplum | MonetDB |
|---|---|---|---|---|---|---|---|
| Storage model | Columnar | Columnar | Columnar | Columnar | Columnar | Columnar (row + col) | Columnar |
| Compression | LZ4, ZSTD, Delta | LZ4, ZSTD | LZ4, ZSTD | LZ4, ZSTD | Snappy, ZSTD | ZSTD, LZ4, Zlib | Lightweight |
| Partitioning | Yes (by expression) | Yes (time-based) | Yes (range, list, expression) | Yes (range, list, hash) | Yes (time, column) | Yes (range, list, hash) | Yes (range) |
| Materialized views | Yes | No (pre-aggregation) | Yes | Yes | No | Yes | No |
| Replication | Yes (multi-shard) | Yes (deep storage) | Yes (multi-replica) | Yes (multi-replica) | Yes (segment) | Yes (mirrors) | No |
| Joins | Yes (distributed) | Limited (lookup) | Yes (distributed) | Yes (distributed) | Limited (lookup) | Yes (MPP) | Yes |
| Real-time ingest | Yes (Kafka, insert) | Yes (Kafka, Kinesis) | Yes (Kafka, routine load) | Yes (Kafka, routine load) | Yes (Kafka, Kinesis) | No (batch ETL) | No (batch) |
| SQL dialect | ClickHouse SQL | Druid SQL | MySQL-compatible | MySQL-compatible | Presto SQL | PostgreSQL-compatible | SQL:2003 |
| Iceberg support | Yes (external) | No | Yes (external catalog) | Yes (external catalog) | No | No | No |
| Cluster mode | Yes (sharded) | Yes (ZooKeeper) | Yes (FE/BE nodes) | Yes (FE/BE nodes) | Yes (Helix/ZK) | Yes (segments) | No (single-node) |
Cloud / Commercial
| Feature | Snowflake | BigQuery | Redshift | Azure Synapse | Databricks | MotherDuck |
|---|---|---|---|---|---|---|
| Storage model | Columnar (micro-partition) | Columnar (Capacitor) | Columnar | Columnar | Delta Lake (Parquet) | Columnar (DuckDB) |
| Compression | Automatic | Automatic | AZ64, LZO, ZSTD | Automatic | Delta + ZSTD | DuckDB native |
| Partitioning | Micro-partitions (auto) | Native + clustering | Distribution + sort keys | Distributions | Delta partitioning | N/A (auto) |
| Materialized views | Yes | Yes | Yes | Yes | Yes (Delta Live Tables) | No |
| Separation of compute | Yes (virtual warehouses) | Yes (serverless) | Yes (RA3 nodes) | Yes (serverless/dedicated) | Yes (clusters) | Partial (hybrid) |
| Time travel | Yes (90 days) | Yes (7 days) | No | Yes (restore points) | Yes (Delta time travel) | No |
| Iceberg support | Yes (Iceberg tables) | Yes (BigLake) | Yes (Spectrum) | Yes (via Fabric) | Yes (UniForm) | No |
| Serverless | Yes (virtual WH) | Yes (native) | Yes (Serverless) | Yes (on-demand) | Yes (SQL warehouses) | Yes (hybrid) |
| Multi-cloud | AWS, Azure, GCP | GCP only | AWS only | Azure only | AWS, Azure, GCP | AWS (DuckDB hybrid) |
Query Engine Comparison
| Feature | Trino | DuckDB | Presto | Dremio | Spark SQL | Starburst | DataFusion |
|---|---|---|---|---|---|---|---|
| Architecture | Distributed MPP | In-process | Distributed MPP | Distributed | Distributed | Distributed MPP | In-process |
| Concurrency | High | Single-process | High | High | Medium | High | Single-process |
| Latency | Sub-second (warm) | Microseconds (local) | Sub-second (warm) | Sub-second | Seconds-minutes | Sub-second | Microseconds |
| Connectors | 50+ (Hive, Iceberg, Delta, MySQL, PG, S3, etc.) | Parquet, CSV, JSON, Iceberg, S3, PG, MySQL | 30+ (similar to Trino) | 20+ (S3, RDBMS, Iceberg, Delta) | Hive, Iceberg, Delta, JDBC, S3 | Trino + enterprise connectors | Parquet, CSV, JSON, S3 |
| Iceberg | Yes (native catalog) | Yes (v2, read/write) | Yes | Yes (native) | Yes | Yes | Yes (read) |
| Delta Lake | Yes (connector) | Yes (read) | Yes | Yes (native) | Yes (native) | Yes | Yes (read) |
| Hudi | Yes (connector) | No | Yes | No | Yes (native) | Yes | No |
| S3/MinIO | Yes | Yes (httpfs) | Yes | Yes | Yes | Yes | Yes (object_store) |
| Federation | Yes (cross-catalog) | No (single-process) | Yes | Yes (virtual datasets) | Limited | Yes (enhanced) | No |
| Cost-based optimizer | Yes | Yes | Yes | Yes | Yes | Yes (enhanced) | Yes |
| Language | Java | C++ | Java | Java | Scala/Java | Java | Rust |
| License | Apache-2.0 | MIT | Apache-2.0 | Apache-2.0 (Community) | Apache-2.0 | Proprietary | Apache-2.0 |
Lakehouse Format Comparison
| Feature | Iceberg | Delta Lake | Hudi | DuckLake | Paimon |
|---|---|---|---|---|---|
| License | Apache-2.0 | Apache-2.0 | Apache-2.0 | MIT | Apache-2.0 |
| ACID transactions | Yes | Yes | Yes | Yes | Yes |
| Time travel | Yes (snapshot) | Yes (version) | Yes (timeline) | Yes | Yes (snapshot) |
| Schema evolution | Yes (full) | Yes (add/rename) | Yes (add/rename) | Yes | Yes |
| Partition evolution | Yes (hidden) | No (requires rewrite) | No | N/A | Yes |
| File format | Parquet, ORC, Avro | Parquet | Parquet, ORC | Parquet | ORC, Parquet |
| Catalog | REST, Hive, AWS Glue, Nessie | Unity, Hive, AWS Glue | Hive, AWS Glue | PostgreSQL, DuckDB | Hive, filesystem |
| Engine support | Trino, DuckDB, Spark, Flink, Presto, Athena, Snowflake, BigQuery, Databricks | Spark, Trino, Flink, DuckDB, Databricks | Spark, Trino, Flink, Presto | DuckDB | Flink, Spark, Trino |
| Streaming ingest | Yes (via Flink) | Yes (Spark Structured Streaming) | Yes (native CDC/upserts) | No | Yes (native Flink) |
| Merge-on-Read | Yes (v2) | Yes (deletion vectors) | Yes (native) | No | Yes |
| Copy-on-Write | Yes | Yes | Yes | Yes | Yes |
| Compaction | Yes (rewrite) | Yes (OPTIMIZE) | Yes (inline/async) | Yes | Yes (auto) |
| Industry momentum | Highest (converging standard) | High (Databricks ecosystem) | Medium (Uber/AWS) | Emerging (DuckDB ecosystem) | Growing (Flink ecosystem) |
ETL / ELT Comparison
Self-Hosted
| Feature | dbt | Airflow | Dagster | Prefect | Meltano | NiFi | Singer |
|---|---|---|---|---|---|---|---|
| Primary role | Transform (T) | Orchestrate | Orchestrate + assets | Orchestrate | ELT (E+L+T) | Data flow | Extract + Load (E+L) |
| License | Apache-2.0 | Apache-2.0 | Apache-2.0 | Apache-2.0 | MIT | Apache-2.0 | AGPL-3.0 |
| Language | SQL + Jinja | Python | Python | Python | Python + YAML | Java (visual) | Python |
| Connectors | 50+ adapters (warehouse targets) | N/A (orchestrates tasks) | N/A (orchestrates) | N/A (orchestrates) | 300+ Singer taps/targets | 300+ processors | 300+ taps/targets |
| Incremental | Yes (incremental models) | Via task logic | Via asset partitions | Via task logic | Via Singer state | Via FlowFile state | Via state messages |
| CDC support | Via sources/snapshots | Via operators | Via sensors | Via tasks | Via CDC taps | Yes (native) | Via CDC taps |
| dbt integration | N/A (is dbt) | Yes (operator) | Yes (asset) | Yes (task) | Yes (native) | No | No |
| Web UI | dbt Docs (static) | Yes (Airflow UI) | Yes (Dagit/Dagster UI) | Yes (Prefect UI) | No (CLI) | Yes (NiFi UI) | No (CLI) |
| Scheduling | Via orchestrator | Yes (cron, sensors) | Yes (schedules, sensors) | Yes (schedules) | Yes (Airflow or cron) | Yes (CRON scheduling) | Via orchestrator |
| Testing | Yes (schema + data tests) | Via task assertions | Yes (asset checks) | Via task assertions | Via dbt tests | No (monitoring) | No |
| Docker | Yes | Yes | Yes | Yes | Yes | Yes | Yes (via Meltano) |
Cloud / Commercial
| Feature | Airbyte | Fivetran | Stitch | Matillion |
|---|---|---|---|---|
| Primary role | Extract + Load (E+L) | Extract + Load (E+L) | Extract + Load (E+L) | ETL + ELT |
| License | ELv2 (Community) | Proprietary | Proprietary | Proprietary |
| Connectors | 350+ | 400+ | 200+ | 100+ |
| Incremental | Yes (CDC, cursor) | Yes (log-based CDC) | Yes (incremental) | Yes |
| dbt integration | Yes (dbt Cloud) | Yes (dbt Core/Cloud) | No | Yes (dbt Core) |
| Normalization | Yes (basic) | Yes (automatic) | Yes (basic) | Yes (transformation UI) |
| Self-hosted | Yes (OSS/Enterprise) | No (SaaS only) | No (SaaS only) | Yes (hybrid) |
| Pricing model | Rows synced | Monthly Active Rows | Rows replicated | Credits |
Data Catalog Comparison
| Feature | DataHub | OpenMetadata |
|---|---|---|
| License | Apache-2.0 | Apache-2.0 |
| Discovery | Yes (search, browse, tags) | Yes (search, browse, tags, tiers) |
| Lineage | Yes (column-level) | Yes (column-level) |
| Governance | Yes (policies, domains, glossary) | Yes (policies, teams, glossary) |
| Ingestion | 50+ sources (push/pull) | 70+ sources (pull-based) |
| Data quality | Via integrations (Great Expectations) | Yes (native test suites) |
| Collaboration | Yes (threads, tasks) | Yes (conversations, tasks) |
| SSO | OIDC (native) | OIDC, SAML (native) |
| Language | Java, Python | Java, Python |
| Architecture | Kafka + Elasticsearch + MySQL/PG | Elasticsearch + MySQL/PG |
| Docker | Yes | Yes |
| Kubernetes | Yes (Helm chart) | Yes (Helm chart) |
Deployment Comparison
| Tool | Self-Hosted | Docker | Kubernetes | Cloud Option | Language |
|---|---|---|---|---|---|
| ClickHouse | Yes | Yes | Yes (Operator) | ClickHouse Cloud | C++ |
| Apache Druid | Yes | Yes | Yes (Helm) | Imply Cloud | Java |
| StarRocks | Yes | Yes | Yes (Operator) | CelerData Cloud | C++/Java |
| Apache Doris | Yes | Yes | Yes (Operator) | SelectDB Cloud | C++/Java |
| Apache Pinot | Yes | Yes | Yes (Helm) | StarTree Cloud | Java |
| Greenplum | Yes | Yes | Yes | VMware Tanzu | C/Python |
| MonetDB | Yes | Yes | No | No | C |
| Snowflake | No | No | No | Snowflake (SaaS) | – |
| BigQuery | No | No | No | BigQuery (GCP) | – |
| Redshift | No | No | No | Redshift (AWS) | – |
| Azure Synapse | No | No | No | Synapse (Azure) | – |
| Databricks | No | No | No | Databricks (multi) | – |
| MotherDuck | No | No | No | MotherDuck (SaaS) | – |
| Trino | Yes | Yes | Yes (Helm) | Starburst Galaxy | Java |
| DuckDB | Yes (embedded) | N/A | N/A | MotherDuck | C++ |
| Presto | Yes | Yes | Yes (Helm) | Ahana Cloud | Java |
| Dremio | Yes | Yes | Yes (Helm) | Dremio Cloud | Java |
| Spark SQL | Yes | Yes | Yes (Operator) | Databricks, EMR | Scala/Java |
| Starburst | Yes | Yes | Yes | Starburst Galaxy | Java |
| DataFusion | Yes (library) | N/A | N/A | No | Rust |
| dbt | Yes (CLI) | Yes | N/A | dbt Cloud | Python |
| Airflow | Yes | Yes | Yes (Helm) | MWAA, Astronomer | Python |
| Dagster | Yes | Yes | Yes (Helm) | Dagster Cloud | Python |
| Prefect | Yes | Yes | Yes | Prefect Cloud | Python |
| Meltano | Yes (CLI) | Yes | N/A | No | Python |
| NiFi | Yes | Yes | Yes (Operator) | Cloudera | Java |
| Singer | Yes (CLI) | Via Meltano | N/A | No | Python |
| Airbyte | Yes | Yes | Yes (Helm) | Airbyte Cloud | Java/Python |
| Fivetran | No | No | No | Fivetran (SaaS) | – |
| Stitch | No | No | No | Stitch (SaaS) | – |
| Matillion | Yes (hybrid) | Yes | Yes | Matillion ETL | Java |
| DataHub | Yes | Yes | Yes (Helm) | Acryl Cloud | Java/Python |
| OpenMetadata | Yes | Yes | Yes (Helm) | Collate Cloud | Java/Python |
Licensing & Pricing
| Tool | License | Pricing Model | Free Tier |
|---|---|---|---|
| ClickHouse | Apache-2.0 | Free (OSS); Cloud: usage-based | Yes (OSS); Cloud free tier |
| Apache Druid | Apache-2.0 | Free (OSS); Imply: usage | Yes (OSS) |
| StarRocks | Apache-2.0 | Free (OSS); CelerData: usage | Yes (OSS) |
| Apache Doris | Apache-2.0 | Free (OSS); SelectDB: usage | Yes (OSS) |
| Apache Pinot | Apache-2.0 | Free (OSS); StarTree: usage | Yes (OSS) |
| Greenplum | Apache-2.0 | Free (OSS) | Yes (OSS) |
| MonetDB | MPL-2.0 | Free (OSS) | Yes (OSS) |
| Snowflake | Proprietary | Credits (compute + storage) | Trial |
| BigQuery | Proprietary | Pay-per-query + storage | 1 TB/month free queries |
| Redshift | Proprietary | Per-node-hour or Serverless | Trial |
| Azure Synapse | Proprietary | Pay-per-query or provisioned | Trial |
| Databricks | Proprietary | DBU (compute units) | Community Edition |
| MotherDuck | Proprietary | Usage-based | Free tier |
| Trino | Apache-2.0 | Free (OSS) | Yes (OSS) |
| DuckDB | MIT | Free (OSS) | Yes (OSS) |
| Presto | Apache-2.0 | Free (OSS) | Yes (OSS) |
| Dremio | Apache-2.0 (Community) | Free (Community); Enterprise | Yes (Community) |
| Spark SQL | Apache-2.0 | Free (OSS) | Yes (OSS) |
| Starburst | Proprietary | Enterprise subscription | Starburst Galaxy free tier |
| DataFusion | Apache-2.0 | Free (OSS) | Yes (OSS) |
| Apache Iceberg | Apache-2.0 | Free (spec + libs) | Yes |
| Delta Lake | Apache-2.0 | Free (spec + libs) | Yes |
| Apache Hudi | Apache-2.0 | Free (spec + libs) | Yes |
| DuckLake | MIT | Free (OSS) | Yes |
| Apache Paimon | Apache-2.0 | Free (spec + libs) | Yes |
| dbt | Apache-2.0 | Free (Core); Cloud: per-seat | Yes (Core) |
| Airflow | Apache-2.0 | Free (OSS) | Yes (OSS) |
| Dagster | Apache-2.0 | Free (OSS); Cloud: per-seat | Yes (OSS) |
| Prefect | Apache-2.0 | Free (OSS); Cloud: per-task | Yes (OSS) |
| Meltano | MIT | Free (OSS) | Yes (OSS) |
| NiFi | Apache-2.0 | Free (OSS) | Yes (OSS) |
| Singer | AGPL-3.0 | Free (OSS) | Yes (OSS) |
| Airbyte | ELv2 | Free (OSS); Cloud: per-row | Yes (OSS) |
| Fivetran | Proprietary | Per Monthly Active Row | Trial |
| Stitch | Proprietary | Per row replicated | Trial |
| Matillion | Proprietary | Per credit | Trial |
| DataHub | Apache-2.0 | Free (OSS); Acryl managed | Yes (OSS) |
| OpenMetadata | Apache-2.0 | Free (OSS); Collate managed | Yes (OSS) |
Tools
38 tools.
Airbyte
Airbyte is the leading data ingestion platform for building EL (extract-load) pipelines. It offers 350+ pre-built connectors covering databases, SaaS APIs, file formats, and event streams.
License: LicenseRef-Airbyte-ELv2 (source-available) · Kind: web · Deploy: docker, k8s, saas · SSO: none
Amazon Redshift
Amazon Redshift is AWS’s fully managed cloud data warehouse service, launched in 2013. It was the first major cloud data warehouse and popularized the concept of elastic, pay-as-you-go analytical databases.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
Apache Airflow
Apache Airflow is the industry-standard workflow orchestrator for data pipelines. Originally created at Airbnb in 2014 and donated to the Apache Software Foundation, Airflow defines pipelines as Python code using DAGs (directed acyclic grap…
License: Apache-2.0 (OSS) · Kind: web · Deploy: native, docker, k8s · SSO: OIDC
Apache DataFusion
Apache DataFusion is an extensible query engine library written in Rust, designed to be embedded inside other applications and systems rather than deployed as a standalone service.
License: Apache-2.0 (OSS) · Kind: web · Deploy: k8s · SSO: none
Apache Doris
Apache Doris is a real-time analytical database with MySQL protocol compatibility, originally developed as Palo at Baidu and donated to the Apache Software Foundation in 2018.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Apache Druid
Apache Druid is a real-time analytics database designed for sub-second OLAP queries on event- driven data. Originally created at Metamarkets and open-sourced in 2012, Druid became an Apache top-level project in 2018.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Apache Hudi
Apache Hudi (Hadoop Upserts Deletes and Incrementals) is an open table format created at Uber to solve the problem of efficiently ingesting and managing rapidly changing datasets on data lakes.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Apache Iceberg
Apache Iceberg is the leading open table format for large-scale analytical datasets, originally created at Netflix and donated to the Apache Software Foundation.
License: Apache-2.0 (OSS) · Kind: web · Deploy: saas · SSO: none
Apache NiFi
Apache NiFi is a visual data flow platform designed for real-time data routing, transformation, and system mediation.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native, docker · SSO: OIDC
Apache Paimon
Apache Paimon is a streaming-first lakehouse table format that originated from the Apache Flink ecosystem. Originally developed as Flink Table Store, it was renamed to Apache Paimon and graduated to an Apache top-level project.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Apache Pinot
Apache Pinot is a real-time distributed OLAP datastore designed for low-latency, high- throughput user-facing analytics.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Apache Spark SQL
Apache Spark SQL is the SQL module of Apache Spark, the general-purpose distributed computing framework for large-scale data processing.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Azure Synapse Analytics
Azure Synapse Analytics is Microsoft’s unified analytics service on Azure, combining enterprise data warehousing with big data analytics in a single platform.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
ClickHouse
ClickHouse is the fastest open-source columnar OLAP database, originally developed at Yandex for web analytics and open-sourced in 2016. It is written entirely in C++ and optimized for real-time analytical queries over billions of rows.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Dagster
Dagster is an open-source data orchestrator built around software-defined assets, offering a visual asset graph, partitions, sensors, type checking, and strong dbt integration for data pipelines.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native, docker, k8s · SSO: none
Databricks
Databricks is the leading lakehouse platform, combining data lake flexibility with data warehouse performance and governance.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: OIDC
DataHub
DataHub is an open-source metadata platform originally developed at LinkedIn and donated to the community under the Apache-2.0 license.
License: Apache-2.0 (OSS) · Kind: web · Deploy: docker, k8s, saas · SSO: none
dbt
dbt (data build tool) is the industry-standard SQL transformation framework for the “T” in ELT pipelines.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native, docker · SSO: none
Delta Lake
Delta Lake is an open table format created by Databricks that adds ACID transactions, scalable metadata handling, and time travel to data stored as Parquet files on object storage.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Dremio
Dremio is a lakehouse query engine built on Apache Arrow that provides SQL analytics directly on data lake storage without requiring data movement or ETL pipelines.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: OIDC
DuckDB
DuckDB is an in-process OLAP database engine, often described as “SQLite for analytics.” Unlike distributed query engines that require cluster infrastructure, DuckDB runs as an embedded library inside the host application process – no serv…
License: MIT (OSS) · Kind: web · Deploy: saas, docker · SSO: none
DuckLake
DuckLake is a lakehouse table format created by DuckDB Labs in 2025 that takes a fundamentally different approach to metadata management.
License: MIT (OSS) · Kind: web · Deploy: saas · SSO: none
Fivetran
Fivetran is the market-leading managed EL (extract-load) platform purpose-built for automated data replication into cloud data warehouses.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
Google BigQuery
Google BigQuery is a fully managed, serverless cloud data warehouse on Google Cloud, using the Dremel engine to query large datasets in SQL, with columnar storage, BigQuery ML, and BigLake governance.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
Greenplum
Greenplum is a massively parallel processing (MPP) analytical database built on PostgreSQL. Originally developed by Greenplum Inc.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Matillion
Matillion is a visual ETL/ELT platform for cloud data warehouses such as Snowflake, BigQuery, Redshift, and Databricks, using pushdown transformation that compiles logic into warehouse-native SQL.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
Meltano
Meltano is a Singer-based ELT platform originally created by GitLab as an internal data integration tool and later spun out as an independent open- source project.
License: MIT (OSS) · Kind: web · Deploy: native, docker · SSO: none
MonetDB
MonetDB is a pioneering columnar analytical database developed at CWI (Centrum Wiskunde & Informatica) in Amsterdam, with research origins dating back to the 1990s.
License: MPL-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
MotherDuck
MotherDuck is a serverless cloud analytics platform built on DuckDB, the open-source in- process analytical database.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
OpenMetadata
OpenMetadata is an open-source metadata platform that provides data discovery, lineage, governance, and data quality capabilities in a single unified platform.
License: Apache-2.0 (OSS) · Kind: web · Deploy: docker, k8s, saas · SSO: none
Prefect
Prefect is a Python-native workflow orchestrator designed to be simpler and more Pythonic than Airflow.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native, docker, k8s · SSO: none
Presto
Presto is a distributed SQL query engine originally created at Facebook (Meta) in 2012 for interactive analytics over the Hive data warehouse.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Singer
Singer is an open-source specification for data extraction and loading. It defines a JSON-based protocol for communication between “taps” (extractors that pull data from sources) and “targets” (loaders that push data to destinations).
License: AGPL-3.0-only (OSS) · Kind: web · Deploy: native, docker · SSO: none
Snowflake
Snowflake is the dominant independent cloud data warehouse platform, operating as a fully managed SaaS service across AWS, Azure, and GCP.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
Starburst
Starburst is the commercial distribution of Trino, created and maintained by the company founded by the original Trino creators.
License: Proprietary (proprietary) · Kind: web · Deploy: native · SSO: OIDC
StarRocks
StarRocks is a high-performance analytical database designed for real-time, sub-second queries on large datasets.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: none
Stitch
Stitch is a managed data replication service owned by Qlik (acquired via Talend). It focuses on simple, reliable EL (extract-load) from SaaS applications and databases into cloud data warehouses.
License: Proprietary (proprietary) · Kind: web · Deploy: saas · SSO: none
Trino
Trino is a distributed SQL query engine designed for interactive analytical queries across massive datasets and heterogeneous data sources.
License: Apache-2.0 (OSS) · Kind: web · Deploy: native · SSO: OIDC