Skip to content

docs: rewrite comparison.md to match ADR 0011's positioning decision (#1053) #1071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
masukai merged 2 commits into from
Sep 1, 2026
Merged

docs: rewrite comparison.md to match ADR 0011's positioning decision (#1053) #1071

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0625e04
docs: rewrite comparison.md to match ADR 0011's positioning decision …
masukai Sep 1, 2026
c6dfc96
fix: correct over-broad hosted-intermediary claim about named competi…
masukai Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- **`docs/guides/comparison.md` rewritten to match ADR 0011's positioning decision** ([#1053](https://github.com/drt-hub/drt/issues/1053)): replaced the line-by-line feature-matrix table against Census/Hightouch/Polytomic (including a losing "22 vs 200+ destinations" row) with ADR 0011's single structural claim — drt never routes data through a drt-hosted intermediary — plus a "why not a UI/hosted runtime/audience builder" FAQ section closing the ADR's own follow-up item. Connector counts in the remaining factual "coverage" section corrected to the current 13 sources / 34 destinations (previously stale at 10/22-23). The tagline/README half of the ADR's follow-up was already shipped in #1010; this closes the remainder.

- **Internal — Snowflake/Databricks join `BaseSqlDestination`'s `finalize_sync()` template, closing the last piece of #720** ([#1030](https://github.com/drt-hub/drt/issues/1030)): `finalize_sync()`'s swap-completion half (guard, shadow-name computation, connection/cursor lifecycle, dispatch) is now shared across Postgres/MySQL/Snowflake/Databricks instead of Snowflake/Databricks each carrying their own full copy. New per-dialect hooks (`_complete_swap`, `_reset_swap_state_after_completion`, `_reset_swap_state_after_noop`) preserve each dialect's exact recovery timing: Snowflake still leaves `_swap_shadow_created`/`_swap_table` intact after a failed `SWAP` so the shadow stays recoverable via `drt clean --orphans`, while Postgres/MySQL/Databricks still reset unconditionally — there's no equivalent recovery path for `INSERT OVERWRITE` to preserve. `_swap_direct_write` (the first-run-wrote-directly-to-target flag, which only exists on Snowflake/Databricks) is reset correctly in both the early no-op guard and the completion path via the same hooks, without being added to Postgres/MySQL's state. Mirror-mode dispatch is untouched — Databricks keeps its own `_finalize_mirror` override per #720's Phase 1 decision. No behavior change to any sync path.

- **The frozen source Protocols now describe third-party profiles honestly** ([#1034](https://github.com/drt-hub/drt/issues/1034)): `Source.extract()` / `Source.test_connection()` and `IncrementalSource.extract_incremental()` previously accepted the closed 13-member `ProfileConfig` union even though [#997](https://github.com/drt-hub/drt/issues/997) / [#1011](https://github.com/drt-hub/drt/pull/1011) had already made plugin-registered source profiles loadable at runtime by force-casting an arbitrary plugin instance into that union. That made the stable Protocol declaration inaccurate at the exact extension boundary ADR 0007 freezes. The methods now accept a runtime-checkable structural `ProfileConfigLike` containing only the two operations drt-core actually uses generically (`type` and `describe()`); all 13 built-in profile dataclasses satisfy it unchanged, and their source implementations keep the established first-line `assert isinstance(config, ConcreteProfile)` narrowing pattern. `load_profile()` now returns that honest structural type and validates the plugin instance instead of casting it, while the generic dispatch/execution path carries the same type through to extraction. A new plugin-authoring guide documents the required method signature and why annotating a plugin method directly with its concrete profile class violates Protocol parameter contravariance. This closes the source-side typing gap left after the plugin config work contributed by [@Muawiya-contact](https://github.com/Muawiya-contact).
Expand Down
121 changes: 78 additions & 43 deletions docs/guides/comparison.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Choosing a Reverse ETL Tool

This page compares approaches to reverse ETL — activating data from your warehouse to external services. We focus on factual differences so you can make the right choice for your stack.
This page explains how drt differs from commercial reverse-ETL tools — not
with a feature checklist, but with the one structural claim that actually
matters, plus the reasoning behind it. See
[ADR 0011](../adr/0011-subtraction-positioning-vs-reverse-etl.md) for the
full decision this page is based on.

## The Modern Data Activation Stack

Expand All @@ -12,45 +16,37 @@ All three are open-source, CLI-first, YAML-configured, and MCP-enabled. Together

---

## Comparison

| | **drt** | **Census** | **Hightouch** | **Polytomic** |
|---|---|---|---|---|
| **Type** | OSS (Apache 2.0) | SaaS (Fivetran) | SaaS | SaaS / self-hosted |
| **Pricing** | Free | Paid plans | Paid plans | Paid plans / free tier |
| **Deployment** | Self-hosted | Managed cloud | Managed cloud | Managed / self-hosted |
| **Configuration** | YAML + CLI | GUI | GUI | GUI |
| **Git-native** | Yes (YAML in repo) | Partial (API) | Partial (API) | Partial (API) |
| **CI/CD** | Native (exit codes, `--output json`) | Webhook/API | Webhook/API | API |
| **Sources** | 10 | 20+ | 30+ | 20+ |
| **Destinations** | 22 | 200+ | 200+ | 50+ |
| **MCP Server** | Yes (6 tools) | Partial (enrichment) | No | Yes |
| **LLM Skills** | Yes (Claude Code) | AI Columns | No | No |
| **Orchestration** | Dagster, Airflow, Prefect | Built-in | Built-in | Built-in |
| **Data validation** | `drt test` (5 validators) | Built-in | Built-in | Limited |
| **Incremental sync** | Cursor-based | Multiple strategies | Multiple strategies | Multiple strategies |
| **Managed infrastructure** | No (you host) | Yes | Yes | Yes / No |

---

## When to choose drt

**drt is a good fit when:**

- You want a **free, self-hosted** solution with no vendor lock-in
- Your team already uses **dbt and/or dlt** and wants the same developer experience for reverse ETL
- You value **Git-native configuration** — YAML files in your repo, reviewed in PRs, deployed via CI
- You work with **AI coding tools** (Claude, Cursor) and want your reverse ETL layer to be accessible via MCP
- You need **10-30 destinations** rather than 200+ — and the ones you need are covered
- You prefer to **own your data pipeline** end-to-end
- You want to grant **read-only access to your source warehouse** — drt's supported incremental strategy (cursor-based, reading a column like `updated_at`) never needs to write into the source *to read from it*. (One opt-in exception: `sync.watermark.storage: bigquery` — see below.)

**drt is not the right fit when:**

- You need **200+ pre-built connectors** out of the box — Census and Hightouch have much larger connector catalogs
- You need a **managed, no-ops solution** — drt requires you to host and maintain the pipeline
- Your team is **non-technical** and prefers a GUI over YAML/CLI
- You need **built-in scheduling and monitoring** — drt relies on external orchestrators (Dagster, Airflow, cron)
## The core difference

**drt has no hosted service to opt out of.** There's no drt-operated
control plane, data plane, or intermediary of any kind — your data goes
straight from your warehouse to the destination, using infrastructure you
already run (GitHub Actions, Dagster, cron, a container). This isn't a
deployment tier or a cost-conscious fallback; it's the only mode drt has.
Commercial reverse-ETL tools (Census, Hightouch, RudderStack Reverse ETL,
Polytomic, and similar) are SaaS-first by default — some offer a
self-hosted or on-prem tier as a secondary option layered onto an
otherwise hosted product, others don't offer one at all; either way,
self-hosting isn't the product's default shape the way it is drt's only
shape.

That one structural fact is also why three other things are true, on
purpose rather than by accident of what hasn't been built yet:

- **No per-row bill.** There's no metered infrastructure standing between
your warehouse and the destination to bill against.
- **No web UI.** Config-as-code — YAML in your repo, reviewed in PRs,
diffable — is the position, not a placeholder for a UI that hasn't been
built yet.
- **No audience/segmentation builder.** Building the record set to sync is
a SQL/dbt-modeling problem. drt reads whatever query or dbt model a
sync's `model` field points at and syncs exactly that — it doesn't
duplicate the modeling layer you already have.

drt does one thing — read a query, map fields, write to a destination API
— and leaves loading, transforming, hosting, and billing to the tools built
for those jobs. This is a deliberate design decision, not a temporary gap:
see the FAQ below for the reasoning and its honest tradeoff.

---

Expand Down Expand Up @@ -90,6 +86,45 @@ project/dataset) to keep the source warehouse completely untouched.

---

## FAQ: why not a UI, a hosted runtime, or an audience builder?

**Isn't a UI table stakes for a reverse-ETL tool?** For a commercial
product, yes — the UI usually *is* the product. drt's position is that
config-as-code is a better fit for a team that already reviews dbt models
and Airflow DAGs in pull requests: a sync is a YAML file, diffable and
git-blame-able like any other change to the pipeline.

**Why no hosted/managed version?** Because that's what makes "no per-row
bill" structural rather than a pricing tier that could change later.
There's no drt-operated service between your warehouse and your
destinations to meter in the first place — drt runs as a process inside
infrastructure you already operate.

**Why doesn't drt have an audience builder?** Because you likely already
have one: dbt. Segmenting "which rows should sync" is a modeling problem,
and duplicating dbt's job inside the sync tool is the same "one tool doing
four jobs" mistake this design avoids elsewhere. Point a sync's `model`
field at a dbt model or raw SQL, and drt syncs exactly what that query
returns.

**Does this mean drt is missing features?** Depends on what you're
comparing. drt covers 13 sources and 34 destinations today (see below) —
fewer than a 200+-connector commercial catalog, and if your workflow
needs one of those, a commercial tool is the better fit. What drt doesn't
have is a UI, a hosted runtime, or an audience builder to catch up on —
those are the position, not gaps waiting to be filled. New destinations
are also a plugin away, not a request to drt-hub: see
[Contributing](https://github.com/drt-hub/drt/blob/main/CONTRIBUTING.md).

**Is this permanent?** For `drt-core` (this repository), yes — a feature
request reintroducing a UI, hosted runtime, or audience builder here would
be declined by default, citing
[ADR 0011](../adr/0011-subtraction-positioning-vs-reverse-etl.md). That
ADR governs this OSS engine specifically, not every product drt-hub might
ever build.

---

## The ecosystem advantage

drt is designed to complement, not compete with, the modern data stack:
Expand Down Expand Up @@ -120,13 +155,13 @@ This means your entire data pipeline — from ingestion to activation — can be

drt currently supports:

**Sources (10):** BigQuery, DuckDB, PostgreSQL, Snowflake, SQLite, Redshift, ClickHouse, MySQL, Databricks, SQL Server
**Sources (13):** BigQuery, DuckDB, PostgreSQL, Snowflake, SQLite, Redshift, ClickHouse, MySQL, Databricks, SQL Server, Delta Lake, Iceberg, REST API

**Destinations (23):** REST API, Slack, Discord, Teams, GitHub Actions, HubSpot, Zendesk, Google Sheets, PostgreSQL, MySQL, ClickHouse, Parquet, CSV/JSON/JSONL, Jira, Linear, SendGrid, Notion, Twilio SMS, Intercom, Email SMTP, Salesforce Bulk API, Google Ads, Staged Upload
**Destinations (34):** REST API, Slack, Discord, Teams, GitHub Actions, HubSpot, Zendesk, Google Sheets, PostgreSQL, MySQL, ClickHouse, Snowflake, Databricks, BigQuery, Parquet, File (CSV/JSON/JSONL), S3, GCS, Azure Blob, Jira, Linear, SendGrid, Notion, Twilio SMS, Intercom, Email SMTP, Salesforce Bulk API, Staged Upload, Google Ads, Amplitude, Mixpanel, Elasticsearch, Airtable, Klaviyo

**Integrations:** Dagster (`dagster-drt`), Airflow (built-in), Prefect (built-in), dbt manifest reader

New connectors are added regularly by the community. The generic REST API destination covers any HTTP endpoint not yet supported natively. See [Good First Issues](https://github.com/drt-hub/drt/issues?q=is%3Aopen+label%3A%22good+first+issue%22) to contribute a connector.
New connectors are added regularly by the community, and third-party connectors can register as a plugin without drt-hub as a gatekeeper. The generic REST API destination covers any HTTP endpoint not yet supported natively. See [Good First Issues](https://github.com/drt-hub/drt/issues?q=is%3Aopen+label%3A%22good+first+issue%22) to contribute a connector.

---

Expand Down
Loading