Is CDC Exactly-Once? Check Each Hop

Read the sentence before you trust it

Debezium's documentation says it provides at-least-once delivery. A vendor page says its service delivers “with exactly-once semantics”. Both can be true, because a guarantee belongs to a hop, not to a pipeline. This page is the companion to Exactly-Once Semantics: that page gives the summary table and the idempotent-consumer patterns, and this one goes down to the settings, shows what each guarantee covers, and ends with how to check what your pipeline gives.

Five questions scope any exactly-once sentence:

1. Which hop? Database to connector, connector to topic, topic to topic, topic to sink, or the sink's own store.
2. Which producer or connector, and is the opt-in switched on? Many guarantees are off by default.
3. Which sink? Does it share a transaction with the offsets, or only receive a call?
4. What happens on failover? A crash, a restart, a rebalance, a second worker that thinks it owns the task.
5. What dedupes, on which key, ordered by what? The answer that matters at the end is a primary key and a source log position, not a timestamp.

The first two phrases come from Streamkap's SQL Server CDC guide and Google Cloud's change data capture explainer (read 2026-10-09). They are examples of wording that leaves the hop and the mechanism open, not claims we found to be wrong. Streamkap's sentence names a destination but no mechanism, and its page does not mention at-least-once delivery or idempotent writes. Google's sentence is about restarting the read from the log and does not mention a destination.

As of: 2026-10-09, against the Debezium 3.7 documentation (the exactly-once page exists for Debezium 3.3 to 3.7 and is absent from 3.2) and the Apache Kafka 4.1 documentation. These pages change; if a setting below does not match your version, trust your version's documentation.

Four hops, and what each guarantee means

Nothing in the table adds up to a single transaction from the database to the sink. The guarantee across independent systems is not achievable as delivery; the achievable thing is effectively-once processing: duplicates may arrive, and the sink makes them harmless.

Hop 2 in practice: the Kafka Connect opt-in

Kafka Connect supports exactly-once for source connectors from version 3.3.0, in distributed mode only (KIP-618, built on the transactions of KIP-98). Debezium's page documents it for the MariaDB, MongoDB, MySQL, Oracle, PostgreSQL and SQL Server connectors. The settings it lists:

# Every Connect worker (distributed mode, Connect 3.3.0 or later)
exactly.once.source.support=enabled

# The connector configuration, added to your existing Debezium settings.
# transaction.boundary is "poll" by default, and Debezium requires poll.
"exactly.once.support": "required",
"transaction.boundary": "poll"

What the framework does when you switch it on

• Records and offsets commit together. With the default poll boundary Connect starts a Kafka transaction for each batch a task returns from poll, writes the batch and its source offsets inside it, and commits. After a crash either both are there or neither is, so a restarted task resumes from an offset that matches what is in the topic.
• Zombie tasks are fenced. Each task gets a transactional ID of the form <group.id>-<connector>-<task id>. A new producer with that ID bars the older one from writing, which stops a task that has been replaced but is still running.
• The check happens before you start. With exactly.once.support=required the framework asks the connector whether it can provide exactly-once with that configuration. If it cannot, or the worker validating the request does not have exactly-once enabled, the request to create or validate the connector fails instead of silently running at-least-once.
• Rolling out is two steps on an existing cluster. Set exactly.once.source.support=preparing on every worker first, then enabled. The workers and the connectors also need ACLs for the transactional IDs and the offsets topic, which Debezium's page leaves to the Kafka documentation.

The framework can only do this for a connector that provides meaningful source offsets for each record and can resume from exactly the position an offset names, which is why the Kafka documentation says exactly-once source support “is highly dependent on the type of connector you run”.

What it does not do

• It covers the connector-to-topic hop. It does nothing for the sink, and the topic you read may still contain duplicates produced by other paths (an upstream replay, a re-snapshot you triggered, a second connector).
• A consumer on the default isolation.level=read_uncommitted returns records from aborted transactions too. A Kafka Connect sink must set consumer.isolation.level=read_committed to benefit.
• A read_committed consumer returns messages only up to the last stable offset, so it sees a batch only after that batch's transaction has committed.

The caveats Debezium itself lists

Debezium's page has a “Known issues and considerations” section. In short: it says it is unclear whether the implementation is fully correct, that no comprehensive study has analysed the correctness of transactions in recent Kafka releases, and that two Jepsen reports (on Redpanda and on Bufstream) raise concerns about the Kafka protocol and its implementations, particularly for Bufstream. It lists three Apache Kafka issues as open and reasons that, since Connect relies on Kafka transactions, they may affect Connect too. Their status in Apache JIRA on 2026-10-09:

The Debezium page lists all three as open, so the page and JIRA disagree on the first. This page does not know whether the two open ones affect a Debezium connector in practice, and neither does Debezium's page: it says it is reasonable to assume they might. That is why the opt-in is a reason to test, not a reason to drop the idempotent sink. KIP-890 describes the delayed-message problem and the server-side defense (a new epoch for each transaction).

Inside Kafka: transactions, offsets and read_committed

If a step reads a topic and writes another topic (a filter, an enrichment, a join), Kafka can make that step exactly-once, and Kafka's design documentation is explicit about where it stops. This hop sits on the same Kafka transactions as the Connect opt-in, so Debezium's reasoning about the open issues above applies to it too.

• The idempotent producer is not enough on its own. enable.idempotence defaults to true in current Kafka and removes duplicates from producer retries, but KIP-98 notes that the guarantee holds within a single producer session: a restarted producer gets a new producer ID.
• A transactional producer (a Kafka transaction) sets transactional.id, which also makes a restarted instance abort its predecessor's in-flight transaction. It writes the output records and calls sendOffsetsToTransaction so the consumer's offsets commit atomically with them (KIP-447 lets the producer pass the consumer's group metadata). The consumer needs isolation.level=read_committed and enable.auto.commit=false.
• Kafka Streams does this for you: processing.guarantee=exactly_once_v2 (the default is at_least_once; v2 needs brokers 2.5 or newer). The documentation scopes it to the input offsets, the state stores and the output topics, committed atomically.

The scope is the sentence to remember: the Kafka documentation says exactly-once delivery “for other destination systems generally requires cooperation with such systems”. A processor that also writes to a database, a warehouse load or an HTTP endpoint is outside the transaction.

Hop 3 and 4: the sink, where it ends

Kafka Connect's documentation says it can provide exactly-once for sink connectors, but it depends on the connector, and the one framework-level instruction is the consumer isolation level. Debezium's JDBC sink says it guarantees events are processed at least once, tells you that after a restart or rebalance it can receive the same event again, and tells you to set insert.mode=upsert to make the writes idempotent. Its delete.enabled setting is off by default, so out of the box a delete event does not remove the row. The page describes upsert as an update-or-insert on the primary key; it does not describe comparing a source log position, so read your sink's documentation for what happens when an older change arrives after a newer one.

Offsets in the sink's transaction

Kafka's own documentation describes the pattern for a destination that has transactions (offsets in the sink transaction): let the consumer store its offset in the same place as its output, because a two-phase commit between the offset store and the output is usually not available. For a relational database, the KafkaConsumer documentation says this lets you commit the result and the offset in a single transaction, and it lists the steps: set enable.auto.commit=false, save the position with each write, and on restart or assignment seek to it. The Kafka design documentation gives a Connect connector that writes data and offsets to HDFS together as its example.

Here is that pattern with a position guard added. The statements use PostgreSQL syntax (INSERT ... ON CONFLICT DO UPDATE ... WHERE); the table and column names are examples. One transaction per Kafka record, or per batch:

CREATE TABLE customers (
  id       bigint PRIMARY KEY,
  name     text,
  deleted  boolean NOT NULL DEFAULT false,   -- a delete is a marker, not a DELETE
  src_lsn  bigint  NOT NULL                  -- source log position of the last applied change
);
CREATE TABLE sink_offsets (
  topic text, part int, next_offset bigint NOT NULL,
  PRIMARY KEY (topic, part)
);

BEGIN;
INSERT INTO customers (id, name, deleted, src_lsn)
VALUES ($1, $2, $3, $4)
ON CONFLICT (id) DO UPDATE
  SET name = EXCLUDED.name, deleted = EXCLUDED.deleted, src_lsn = EXCLUDED.src_lsn
  WHERE customers.src_lsn < EXCLUDED.src_lsn;      -- older or equal position: no-op
INSERT INTO sink_offsets (topic, part, next_offset)
VALUES ($5, $6, $7)                                  -- the record's offset + 1
ON CONFLICT (topic, part) DO UPDATE SET next_offset = EXCLUDED.next_offset;
COMMIT;

On start or partition assignment, read next_offset from sink_offsets and seek there. Run on one customer key, the same two statements behave like this (this repository's tests run the statements as printed (the placeholder numbers differ and comments and the final COMMIT are stripped) on PGlite 0.5.8, which is PostgreSQL 18.3 compiled to WebAssembly, and the logic in SQLite 3.54, which accepts the same ON CONFLICT ... WHERE form; the table below matched in both. They were not run on a PostgreSQL server by this repository):

The strict < makes a replay of the same change (same position) a no-op. If two different changes to one key can share a position, compare the (position, seq) pair instead; the merge cookbook shows that form.

The two guards cover different failures. Storing the offset with the row means a consumer that crashes between the write and the offset commit cannot leave them out of step: the transaction rolled back or it did not. It does not help with a duplicate that is already in the topic at a different offset, because that is a different Kafka record. The position guard is what turns that into a no-op, and it also covers backfills and replays that rewind the consumer. That is why the idempotent, position-ordered upsert stays required even where the first hop is exactly-once. For deletes, keep the row with deleted = true so an older change cannot bring it back; the merge cookbook and materialization pages show the same guard in warehouse MERGE form.

What guarantee do I really have?

Answer these in order; each one says what you can claim.

1. Is there a step that reads a topic and writes a topic, using a transactional producer or Kafka Streams with exactly_once_v2? Then that step is exactly-once with respect to its input and output topics, as long as every consumer of the output reads read_committed, and subject to the open Kafka transaction issues listed above. Claim that, and nothing past the topic. If you have no such step (the connector writes a topic and a plain consumer reads it), there is nothing exactly-once to claim between the topic and the consumer; go to 2.
2. Is the destination an external system (database, warehouse, search index, HTTP call)? Yes: you do not have exactly-once delivery. Go to 3. No, the destination is another Kafka topic: you have whatever step 1 gave you and the connector hop in step 4; skip to 4.
3. Can the destination hold your consumer offsets in the same transaction as the data? If yes, a consumer crash cannot double-apply or skip a record. If no, a crash between the write and the offset commit repeats a record, and only the guard in step 5 protects you.
4. Is the connector-to-topic opt-in off, or on? Off: the topic can contain duplicates after a connector crash. On: fewer, with the caveats above; still assume some.
5. Is the write an upsert on the primary key, applied only if the incoming source log position is newer, with deletes kept as markers? If yes, you have exactly-once processing for this sink: any duplicate or replay leaves the same table. If no, you have whatever the sink's default is, and an older change can overwrite a newer one.

Step 5 is the one that holds regardless of the answers to 3 and 4, which is why the rest of this site treats at-least-once as the default and the idempotent sink as the guarantee.

Verify what your pipeline gives

A configuration that says “required” does not prove behaviour. Three checks, in the order of cost. None of these ran against a live Debezium and Kafka stack for this page; the commands are written from the Kafka tool documentation, and the jq pipeline was run on a small hand-made sample only.

1. Look for duplicates in the topic. Duplicates from the source show up as the same key and the same source position at two different offsets. With the JSON converter and schemas enabled the path is .payload, and .payload.source.lsn is PostgreSQL's position field; substitute your connector's (binlog file and position for MySQL, commit_lsn for SQL Server, scn for Oracle) and adjust the path for your converter. The console consumer prints a tombstone as the literal text null, which the filter skips.

   kafka-console-consumer.sh --bootstrap-server localhost:9092 \
     --topic dbserver1.public.customers --from-beginning --timeout-ms 10000 \
     --isolation-level read_committed \
   | jq -R -r 'fromjson? | select(.payload.source.lsn != null)
       | [.payload.source.lsn, .payload.op, (.payload.after.id // .payload.before.id)] | @tsv' \
   | sort | uniq -d

   Every line printed is a change that appears more than once. Run it again with --isolation-level read_uncommitted to see what a default consumer sees: with the opt-in on, the difference is records from aborted transactions.
2. Inject the duplicates yourself. Stop the sink connector, rewind its consumer group, start it and compare the table with a copy taken before. The Kafka documentation describes kafka-consumer-groups.sh --reset-offsets --to-earliest --execute (the consumer instances must be inactive). If your sink stores offsets itself, as in the pattern above, also reset or delete its sink_offsets row, otherwise the consumer seeks to the stored offset and nothing is replayed. If the table changes, the sink is not idempotent. This is the cheapest test of the claim that matters.
3. Kill the connector mid-batch. Send SIGKILL to the Connect worker while changes are flowing, restart it, and run check 1. With the opt-in off you expect duplicates if the kill landed between writing records and recording offsets; with it on you would expect none in the typical case, but Debezium itself does not claim that holds, so a duplicate here is a finding, not proof the sink may be non-idempotent. In the opt-in-off run, fail the test if you injected no duplicates, since then the sink was never exercised. The Kafka + Debezium lab gives you a stack to try it on, and the offsets and replays page covers rewinds.

See the failure and the fix

The interactive demo on the Exactly-Once page steps through a consumer that writes to the sink, crashes before committing its offset, and re-processes the message, then shows the idempotent version. It demonstrates the sink side of hop 3; it does not simulate the Kafka Connect opt-in.

To see the same sink-side idea on a change feed, open Replay against a guarded sink in the playground: the log sends two changes a second time, and a position guard skips them. It models one lane and one key and does not make delivery exactly-once. The playground is a separate tool that contacts a third-party database (see the privacy note).

Further resources

• Exactly-Once Semantics: the summary table, the demo, idempotent consumers and the outbox.
• Transactional Outbox and Relay: the outbox hop in detail.
• Errata: corrections to wording about exactly-once and effectively-once.
• Event envelope and Offsets & replays: the position fields and rewinds.
• Glossary: Kafka transaction, fencing, offsets in the sink transaction, at-least-once, deduplication, upsert, idempotent write.
• Debezium: Exactly-once delivery: scope, prerequisites, connectors, known issues.
• Debezium FAQ: why consumers must expect duplicate events.
• Debezium JDBC sink: at-least-once delivery, idempotent writes, delete mode.
• Kafka Connect user guide: exactly-once support, worker settings, ACLs, sink isolation level.
• Kafka Connect connector development guide: exactly-once source connectors, transaction boundaries, preflight validation.
• Kafka design: message delivery semantics and using transactions.
• KafkaConsumer: storing offsets outside Kafka.
• KafkaProducer: sendOffsetsToTransaction.
• Kafka Streams core concepts and configuration: processing guarantees and processing.guarantee.
• Kafka basic operations: resetting consumer group offsets.
• KIP-98, KIP-447, KIP-618, KIP-890.
• PostgreSQL INSERT: ON CONFLICT DO UPDATE ... WHERE.