rustcdc

Getting started

Install rustcdc, write a minimal configuration, and stream your first row-level change from PostgreSQL, MySQL, MariaDB or SQL Server in about ten minutes.

On this page

To see it working before configuring anything, run the self-contained demo. It streams live PostgreSQL changes to your terminal in under two minutes: cd demo && docker compose up

You need a database you can enable replication on, and either Rust or Docker. Pick the install path that matches where this will run — a binary for local development, VMs and bare metal, or the container image for Kubernetes and CI.

Install with Homebrew

On macOS (Apple Silicon) and Linux (x86_64, arm64):

brew install hupe1980/tap/rustcdc
rustcdc --version

The formula installs the release binary built with every connector and sink.

Install the binary

Requires Rust 1.97.0 or later (rustup update stable).

git clone https://github.com/hupe1980/rustcdc
cd rustcdc
cargo build --release --locked
sudo cp target/release/rustcdc /usr/local/bin/
rustcdc --version

Building from source needs cmake, clang and perl on the build host — they are required by aws-lc-sys, the cryptography backend used for TLS.

Choosing connectors

A source build captures PostgreSQL only. Connectors are cargo features, and the default is postgres:

FeatureConnectorsDefault
postgresPostgreSQL logical replicationyes
mysqlMySQL and MariaDB binlogno
sqlserverSQL Server CDCno
glueAWS Glue Schema Registry codecno
# MySQL and MariaDB instead of PostgreSQL
cargo build --release --locked --no-default-features --features mysql

# Everything
cargo build --release --locked --all-features

At least one connector is required; a build with none fails to compile rather than producing a binary that rejects every configuration.

A connector you leave out is a driver you do not link, audit or patch. Every build links exactly one TLS stack — one rustls, one rustls-webpki — whichever connectors it has, and a test asserts it for the default build and for the all-features build the image ships.

A config naming a connector the binary lacks is rejected at startup with the feature to rebuild with — it never loads silently.

The container image is built with --all-features and carries every connector, so none of this applies if you deploy the image.

Run with Docker

The official image is multi-arch (linux/amd64, linux/arm64), runs as a non-root user, and is based on distroless/cc — no shell, no package manager.

Pull

docker pull ghcr.io/hupe1980/rustcdc-server:latest

Verify

docker run --rm ghcr.io/hupe1980/rustcdc-server:latest --version

Prepare PostgreSQL

rustcdc reads from the PostgreSQL logical replication stream. You need wal_level = logical and a replication user.

-- 1. Enable logical replication (requires superuser; needs a server restart
--    if wal_level was not already "logical")
ALTER SYSTEM SET wal_level = logical;
SELECT pg_reload_conf();   -- confirm with: SHOW wal_level;

-- 2. Create a dedicated replication user
CREATE USER cdc_user WITH REPLICATION LOGIN PASSWORD 'changeme';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO cdc_user;

-- 3. Create a publication for the tables you want to capture
CREATE PUBLICATION cdc_pub FOR TABLE public.orders, public.customers;

Cloud databases: RDS, Azure Database for PostgreSQL, and Cloud SQL all support logical replication — see the PostgreSQL connector guide.

Create your config

Generate a starter file:

rustcdc init --output cdc.toml --profile dev

Then edit cdc.toml to match your environment. The minimal working config:

api_version = "v1"

[source.postgres]
host        = "localhost"
port        = 5432
user        = "cdc_user"
password    = { env = "POSTGRES_PASSWORD" }   # never hardcode secrets
database    = "mydb"
publication_name      = "cdc_pub"
replication_slot_name = "cdc_slot"
table_include_list = ["public.orders", "public.customers"]
table_exclude_list = []
conn_timeout_secs       = 10
stream_poll_interval_ms = 100
max_events_per_poll     = 1000

# Quickstart convenience: create the replication slot on first connect.
# In production, provision the slot out of band and set this to false —
# a slot that vanishes mid-life is a data-loss event, and recreating it
# silently would skip everything in between.
create_replication_slot_if_missing = true

[source.postgres.transport]
mode = "plaintext"

[sink]
type = "stdout"   # swap for http, kafka, or iceberg when ready

[state]
dir = "./state"

Validate before running:

rustcdc validate-config --config-file cdc.toml

First run

Binary

export POSTGRES_PASSWORD="changeme"

# (Kafka-topic state backend only) seed the state topic first:
# rustcdc init-state --config-file cdc.toml

# Start streaming — the slot is created on first connect thanks to
# create_replication_slot_if_missing = true
rustcdc run --config-file cdc.toml

Docker

docker run --rm \
  -e POSTGRES_PASSWORD="changeme" \
  -v "$PWD/cdc.toml:/etc/rustcdc/config.toml:ro" \
  -v "$PWD/state:/var/lib/rustcdc/state" \
  -p 8080:8080 \
  ghcr.io/hupe1980/rustcdc-server:latest \
  run --config-file /etc/rustcdc/config.toml

Tip: add --network host (Linux) or configure host.docker.internal so the container can reach your local Postgres.

Docker Compose (development)

# compose.yml
services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: changeme
      POSTGRES_DB: mydb
    command: ["postgres", "-c", "wal_level=logical"]
    ports: ["5432:5432"]

  rustcdc:
    image: ghcr.io/hupe1980/rustcdc-server:latest
    depends_on: [postgres]
    environment:
      POSTGRES_PASSWORD: changeme
    volumes:
      - ./cdc.toml:/etc/rustcdc/config.toml:ro
      - rustcdc-state:/var/lib/rustcdc/state
    ports: ["8080:8080"]
    command: ["run", "--config-file", "/etc/rustcdc/config.toml"]

volumes:
  rustcdc-state:
docker compose up

Verify it's working

Events stream to stdout as JSON lines — the full rustcdc event envelope (abridged; real events also carry source, ts, transaction, and envelope_version):

{"before":null,"after":{"id":42,"amount":"99.99","status":"pending"},"op":"insert",
 "schema":"public","table":"orders","primary_key":["id"],"before_is_key_only":false}

Check the admin API (available on port 8080). With the dev profile, /healthz, /livez, and /readyz need no token; /status and /metrics always require a read token — configure [admin] read_token_env and export the variable first:

# Liveness / readiness (no token needed with the dev profile)
curl http://localhost:8080/livez
curl http://localhost:8080/readyz

# Pipeline status and Prometheus metrics (read token required)
export RUSTCDC_ADMIN_READ_TOKEN="my-dev-read-token"   # matches [admin] read_token_env
curl -H "Authorization: Bearer $RUSTCDC_ADMIN_READ_TOKEN" http://localhost:8080/status
curl -H "Authorization: Bearer $RUSTCDC_ADMIN_READ_TOKEN" http://localhost:8080/metrics

Add a transform (optional)

rustcdc can modify, filter, or enrich every event before it reaches the sink. Two modes are available:

Native rules — zero-code, declared in TOML. Good for filtering and simple field operations:

# Drop audit-log events and stamp provenance metadata onto customer events
[[pipeline.transforms]]
name = "drop-audit-log"

  [[pipeline.transforms.actions]]
  type = "filter"
  include_tables = ["customers", "orders"]   # everything else is dropped

[[pipeline.transforms]]
name = "stamp-metadata"

  [pipeline.transforms.when]
  tables = ["customers"]
  ops    = ["insert", "update", "read"]

  [[pipeline.transforms.actions]]
  type         = "metadata_projection"
  target_field = "_meta"
  fields       = ["source_name", "offset", "table", "operation"]

Available native actions: unwrap, flatten, filter, route, metadata_projection, key_shaping. For column redaction/masking, use a WASM module (below).

WASM module — arbitrary logic compiled from Rust, AssemblyScript, TinyGo, or any wasm32 target. Use this for enrichment, complex routing, custom pseudonymisation, and anything native rules cannot express:

[pipeline.transform_runtime]
mode = "wasm"

  [pipeline.transform_runtime.wasm]
  module_path        = "/etc/rustcdc/my_transform.wasm"
  timeout_ms         = 50
  max_memory_bytes   = 8388608
  instance_pool_size = 4

→ Writing WASM transforms for build steps, patterns, testing, and Kubernetes deployment.

Production checklist

Before going to production, work through these items:

  • Sink — swap stdout for kafka, http, or iceberg (sink configuration)
  • Kafka topic layout — one topic, or topic = "cdc.${schema}.${table}" for one per table. Create them first; rustcdc does not (topic naming)
  • Delivery contract — set delivery_contract = "at_least_once" (default) or "effectively_once" for Kafka (delivery contracts)
  • State backend — replace local_fs with kafka_topic, redis, or postgresql for HA (state backends)
  • Secrets — all passwords via { env = "VAR" }, never inline
  • Audit trail — enable signing and IP pseudonymisation (security hardening)
  • Health probes — wire /livez and /readyz to your orchestrator (admin API)
  • SLO alerts — deploy monitoring/rustcdc_slo_alerts.yml to Prometheus

Next steps

GuideWhat it covers
PostgreSQL connectorPublications, slots, replica identity, RDS/Azure/GCP, HA failover
MySQL / MariaDB connectorbinlog_format, server_id, grants, GTID
SQL Server connectorsp_cdc_enable_db, permissions, polling interval
Core conceptsEvent model, pipeline lifecycle, delivery contracts, circuit breaker
Configuration referenceEvery TOML field with defaults and examples
Operations guideReplay, migrate-state, graceful shutdown, performance tuning
Writing WASM transformsWrite custom transforms in Rust, AssemblyScript, or any WASM language

Improve this page on GitHub