Advanced

Transactional Outbox and Relay

The outbox makes the event and the state change commit together. It does not make the publish exactly-once. See why dual writes fail, what the relay and Debezium's router do on a restart, and what a consumer must keep to stay correct.

The question

A service has to change its own database and tell the rest of the system about it. That is two writes to two systems, and they can disagree. The transactional outbox removes the disagreement on the producing side. What it leaves you is a relay that may publish an event more than once, and a consumer that has to cope with duplicates and with order.

CDC delivery is at-least-once, and end-to-end exactly-once across independent systems is not achievable in general. What you can reach is exactly-once processing (see effectively-once): at-least-once publishing into an idempotent consumer keyed on the aggregate id and ordered by a version or log position, with deletes kept as markers. The outbox gives you an atomic intent to publish. It does not give you exactly-once delivery.

The pattern and its diagram are already on Exactly-Once Semantics, and Is CDC Exactly-Once? scopes the claim hop by hop. This page does not repeat them. It adds what is thin there: the table and the transaction as runnable SQL, the two kinds of relay (a polling publisher and a log-tailing connector), Debezium’s Outbox Event Router and what it keeps and drops, what an id can and cannot do for ordering, and when an outbox row can be deleted.

Everything here was checked by reading the Debezium, Kafka, PostgreSQL and microservices.io documentation on 2026-10-09 (links at the end). The SQL was run on PGlite and on SQLite (see what was run). Debezium and Kafka were not run. Where a behaviour is not stated in the pages read, the text says “check your version” instead of guessing.

Why a dual write fails

A dual write changes the database and publishes to the broker as two separate steps. Whatever order you pick, a failure between the steps leaves the two disagreeing.

Four ways to combine the database write and the publish
Approach What can go wrong Result
Publish, then commit The transaction rolls back (a constraint, a crash) after the message was sent. An event for a change that never happened.
Commit, then publish The process crashes, or the broker is unreachable, after the commit. A change with no event. Nothing records that the send is owed.
Commit, then retry the publish in memory The retry state lives in the process and dies with it. A retry after an unacknowledged send can also deliver twice. The same loss as the row above, plus possible duplicates.
One distributed transaction (2PC) over both The database or the broker may not support it, and it couples the service to both. Usually not an option (see the sources below).

The Debezium blog on the pattern puts the last row this way: there cannot be one shared transaction spanning the database and Kafka, because Kafka cannot be enlisted in distributed (XA) transactions. microservices.io gives the same two reasons, and states the requirement: the service must update the database and send the message atomically, with the message sent if and only if the transaction commits. That quote is from 2019 for Kafka; check your broker if you are not on Kafka.

The outbox table and the one transaction

The fix is to modify only one system synchronously. Write the event to an outbox table in the service’s own database, in the same transaction as the state change. If the transaction commits, the event row exists and a separate process (the relay) will send it. If it rolls back, neither exists.

The columns below are the ones Debezium’s router expects by default (id, aggregatetype, aggregateid, type, payload), plus one of mine: seq, a version number for the aggregate. Why it is there is the subject of the consumer section.

Outbox columns and what each is for
Column Holds Used for
id A unique id for this event (a UUID here). Identifying and de-duplicating one event. Not an order.
aggregatetype The kind of aggregate root, such as order. By default, picks the topic.
aggregateid The id of the aggregate root the event belongs to. The Kafka message key, so one aggregate’s events share a partition.
seq The aggregate’s version after this change. Ordering and stale-event detection on the consumer.
type The event type, such as OrderShipped. Letting consumers pick a handler.
payload The event body as jsonb. The message value.
PostgreSQL: the tables and the transaction (run on PGlite)
CREATE TABLE orders (
  id      text   PRIMARY KEY,
  status  text   NOT NULL,
  version bigint NOT NULL DEFAULT 1
);

CREATE TABLE outbox (
  id            uuid   PRIMARY KEY,
  aggregatetype text   NOT NULL,
  aggregateid   text   NOT NULL,
  seq           bigint NOT NULL,
  type          text   NOT NULL,
  payload       jsonb  NOT NULL
);

-- 1. Create the order and its first event together.
BEGIN;
INSERT INTO orders (id, status) VALUES ('ORD-221', 'created');
INSERT INTO outbox (id, aggregatetype, aggregateid, seq, type, payload)
VALUES (gen_random_uuid(), 'order', 'ORD-221', 1, 'OrderCreated',
        '{"status": "created"}');
COMMIT;

-- 2. Change the order and write the event. The UPDATE takes the row lock and
--    returns the new version; the event is written in the same statement.
BEGIN;
WITH bumped AS (
  UPDATE orders
     SET status = 'shipped', version = version + 1
   WHERE id = 'ORD-221'
  RETURNING id, status, version
)
INSERT INTO outbox (id, aggregatetype, aggregateid, seq, type, payload)
SELECT gen_random_uuid(), 'order', id, version, 'OrderShipped',
       jsonb_build_object('status', status)
  FROM bumped;
COMMIT;

Three details carry the weight. First, the version comes from the UPDATE ... RETURNING, not from a read-then-write: under the default READ COMMITTED level a second writer to the same row waits for the first to commit and then increments the updated row, so concurrent writers to one aggregate get consecutive versions in commit order (PostgreSQL documents this wait-and-re-evaluate behaviour; at REPEATABLE READ the second writer gets a serialization error instead and must retry). Second, a rollback undoes the state change, the version bump and the event row together; on PGlite a rolled-back UPDATE left orders at version 2 and outbox at two rows. Third, the application writes the event as a normal, explicit row. The payload is a contract you choose, not a copy of whatever columns changed, which is one reason to prefer it over capturing the business tables directly.

The relay: polling publisher or log tailing

The message relay moves committed outbox rows to the broker. microservices.io names two ways to build it, and lists the drawbacks of each in the pattern pages.

Polling publisher and transaction-log tailing (benefits and drawbacks as listed by microservices.io)
Aspect Polling publisher Log tailing (for example Debezium)
What it reads The outbox table, with a query. The database transaction log (the WAL in PostgreSQL, the binlog in MySQL).
Benefits listed Works with any SQL database. No 2PC; “guaranteed to be accurate”.
Drawbacks listed Tricky to publish events in order. Not all NoSQL databases support it. Relatively obscure; needs database-specific solutions; “tricky to avoid duplicate publishing”.
Where the duplicate comes from A crash after publishing and before recording that it did. A restart that resumes from the last recorded position, which can be earlier than the last event sent.
Cleanup You delete published rows. The row can be deleted at once; see cleanup.

Both are at-least-once. microservices.io states it for the pattern: the relay might publish a message more than once, for example by crashing after publishing and before recording the fact, and a consumer must therefore be idempotent, perhaps by tracking the ids of messages it has already processed.

A polling publisher, and where it duplicates

PostgreSQL: one poll cycle (run on PGlite)
ALTER TABLE outbox ADD COLUMN published_at timestamptz;

-- Read pending events. Within one aggregate they come out in version order.
SELECT id, aggregateid, seq, type, payload
  FROM outbox
 WHERE published_at IS NULL
 ORDER BY aggregateid, seq
 LIMIT 100;

-- ... send each event to Kafka and wait for the broker to acknowledge it ...

-- Record that it was sent. A crash before this line means the next poll
-- returns the same rows again: that is the duplicate.
UPDATE outbox SET published_at = now() WHERE id = :event_id;

The window between the send and the UPDATE is the duplicate. Run the SELECT again without the UPDATE and you get the same rows. Two rules keep the polling publisher honest about order: run one poller (or give each poller a fixed share of the aggregates), and on a failed send for an aggregate, stop publishing that aggregate’s later events until it succeeds. A second poller that skips locked rows, or a loop that carries on past a failure, publishes seq 6 before seq 5.

Do not poll on a global counter such as WHERE id > :last_seen_id with an auto-increment id. A sequence value is assigned at insert, but the row becomes visible at commit, so a transaction with a lower id can commit after the poller has moved past it and its event is never read. A reviewer reproduced this on PostgreSQL 18 (not by me, not on PGlite): with id 1 uncommitted and id 2 committed, a poller asking for id > 0 saw only id 2. The per-aggregate version above does not have this problem for events of one aggregate, because a transaction cannot get version N+1 until version N has committed.

Debezium’s Outbox Event Router

With log tailing, the relay is a Debezium connector plus one single message transform (SMT), io.debezium.transforms.outbox.EventRouter. The connector reads the insert on the outbox table from the log and the SMT turns that change event into the message you want: a topic chosen from the row, a key, a header carrying the event id, and the payload as the value. The connector should capture the outbox table only, and the transform should apply only to its events (the reference suggests an SMT predicate, or route.topic.regex), because heartbeats, schema changes and transaction messages do not have the outbox structure.

The router options this page uses, with the defaults in the Debezium reference
Option Default Effect
table.field.event.id id The column holding the unique event id. It is put in a message header named id, and the reference says you can use it to remove duplicate messages.
table.field.event.key aggregateid The column used as the Kafka message key. The reference calls this important for correct order in Kafka partitions.
table.field.event.payload payload The column used as the message value. With additional fields configured, the value becomes an envelope around the payload and those fields.
table.field.event.timestamp none (the Debezium event time) A column to use as the Kafka message timestamp instead. It labels the message; it is not an ordering key.
route.by.field aggregatetype The column whose value becomes part of the topic name.
route.topic.replacement outbox.event.${routedByValue} The topic name pattern; ${routedByValue} is replaced by the routing column’s value.
table.expand.json.payload false Parse a JSON string payload into real JSON instead of an escaped string. Needs a JSON column and the JsonConverter.
table.fields.additional.placement none Extra outbox columns to put in a header or the envelope, as column:placement[:alias]. The reference also shows a partition placement; check your version.
table.op.invalid.behavior warn What to do when the outbox table is updated: warn, error (both skip the record) or fatal (stop the connector).
tracing.span.context.field tracingspancontext The column holding a tracing span context. Two more options, tracing.operation.name and tracing.with.context.field.only, control the span. Tracing is optional.
Kafka Connect: the router in a connector configuration (not run)
# Connection settings omitted. Capture the outbox table only.
connector.class=io.debezium.connector.postgresql.PostgresConnector
table.include.list=public.outbox
tombstones.on.delete=false

transforms=outbox
transforms.outbox.type=io.debezium.transforms.outbox.EventRouter
transforms.outbox.route.by.field=aggregatetype
transforms.outbox.route.topic.replacement=outbox.event.${routedByValue}
transforms.outbox.table.field.event.key=aggregateid
transforms.outbox.table.expand.json.payload=true
transforms.outbox.table.fields.additional.placement=seq:header:seq

value.converter=org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable=false

That configuration sends the OrderShipped row above to the topic outbox.event.order with the key ORD-221, the event id in the id header, the version in a seq header and the JSON payload as the value. seq:header:seq uses the documented column:placement:alias form; seq:envelope:seq would put it in the value instead. Either way, test what your consumer actually sees: a header reaches a plain consumer as bytes. With value.converter.schemas.enable=false the value is the bare JSON; Kafka Connect’s JsonConverter otherwise defaults to wrapping each value in a schema and payload envelope (check your version), which your consumer would then have to unwrap.

What the router keeps, drops and refuses

  • Keeps. The key, the payload, the event id (as a header), the topic route, any additional columns you name, and a timestamp.
  • Drops deletes. The reference says the SMT automatically filters out DELETE operations on an outbox table. The 3.0.0.Final and 3.4.0.Final sources I read also drop null-valued records (tombstones); the separate route.tombstone.on.empty.payload option (default false) is about a row whose payload is empty, which can emit a tombstone on purpose.
  • Passes snapshot reads through. In those two sources only deletes, tombstones and updates are dropped; op=r (snapshot read) and inserts go on to become messages. So a snapshot of an outbox table that still holds rows re-publishes every retained row, with the same id and seq at new offsets (3.0.0.Final and 3.4.0.Final source; the class moved in later releases and I did not read it there, so check your version).
  • Refuses updates. The reference says all changes to an outbox table are expected to be inserts: the table is a queue. An update is handled by table.op.invalid.behavior. So a log-tailing relay and a polling-style published_at flag do not mix.
  • Does not forward the source position. The reference’s example shows the raw change event with source.lsn and source.txId, and the routed message without them. Reading apply() in the 3.0.0.Final source agrees: the new record is built from the outbox row’s columns, not the source block. If you want a position on the consumer side, put it in a column. This matters in the consumer section.
  • One structure per connector. A connector can capture more than one outbox table only if each has the same structure.
  • Not for MongoDB. The reference says this SMT is not compatible with the MongoDB connector, which has its own outbox router.

Versions. The Debezium documentation I read (the stable pages, 3.7) lists the router as a standard SMT. The options in the table above appear in the reference pages for 1.9, 2.7, 3.0, 3.3 and 3.7. Some names changed: the option for a missing additional field is table.fields.additional.error.on.missing in 2.7 to 3.3 and table.field.additional.missing in 3.7, and a few options (the JSON null and empty-array handling) were added along the way. I read the source of 3.0.0.Final, not 3.7, so the behaviours above taken from the source are “check your version”.

What at-least-once means here

The outbox makes the event durable and atomic with the state change. Publishing it is a separate step with its own failures, and each failure repeats events rather than losing them.

Where a repeat of the same outbox event comes from, and what stays the same
Cause Same event id Same seq Same Kafka offset
Connector restart or Kafka Connect task crash: it resumes from the last recorded offset and re-reads events it already sent Yes Yes No, a new send at a higher offset
Slot position persisted only at a checkpoint, so the slot returns to an earlier position after a database crash Yes Yes No
Polling publisher crashes between the send and marking the row published Yes Yes No
Snapshot or incremental snapshot of an outbox table that still holds rows (first start, re-snapshot after slot loss, an incremental snapshot signal) Yes Yes No, a new send. The router passes snapshot reads through (see what the router drops).
Consumer crashes after processing and before committing its offset Yes Yes Yes, the same message is read again

The first row is documented for the Debezium PostgreSQL connector: when a Kafka Connect process crashes, its tasks stop without recording their latest offsets, replacement tasks resume from the last recorded offset, and they “might generate some of the same change events”. The page tells consumers to anticipate duplicates, and to track the position, especially the LSN, to recognise one. The second row is PostgreSQL’s own: a logical slot emits each change once in normal operation, but its position is persisted only at checkpoint, so after a crash recent changes can be sent again, and decoding clients are responsible for avoiding ill effects. The Debezium blog on the outbox pattern calls its pipeline “at least once” and passes the event id as a header for the same reason.

The column to read is the last one. Only the consumer’s own redelivery repeats an offset. A repeat from the relay or the connector arrives at a new, higher offset with the same id and the same seq. That is why an offset is useless as a version here (see Making sinks bulletproof), and why it matters for pruning, below. A replay after an offset reset is the same thing at a larger scale; the mechanics are on Offsets & replays.

Kafka’s producer settings help only inside one producer session: with enable.idempotence the producer writes one copy of each message despite its own retries, and ordering is preserved with up to five in-flight requests. A connector that re-reads an event after a restart sends it as a new message, which idempotence does not stop.

The consumer: dedupe by id, order by version

Two jobs are easy to merge and are not the same job. Deduplication answers “have I seen this event?”. Ordering answers “is this event newer than what I hold?”. An event id does the first. It does nothing for the second, and a random UUID has no order at all.

What each candidate can tell a consumer about an outbox event
Candidate Detects a repeat Says which is newer Comment
Event id (UUID) Yes, if you remember it No A random id has no order. Needs a store of seen ids and a retention rule.
Global auto-increment id Yes Not reliably Assigned at insert, visible at commit. It also leaves gaps: on PGlite a rolled-back nextval consumed a value, so a gap is not a missing event.
Message timestamp or table.field.event.timestamp No No Can tie, can be skewed, and on a replay may describe when the connector read the row.
Kafka offset Only the consumer’s own redelivery No A relay or connector repeat gets a new, higher offset.
Source log position of the outbox insert Yes Across transactions, in commit order; within one aggregate only if the lock is taken first Not forwarded by the router. See below.
Aggregate id plus seq written in the same transaction Yes Yes, per aggregate Survives the router (as a column you place), a replay, a different relay and a failover.

What to carry

Carry the aggregate id (as the message key) and a version that the producing transaction assigns under the aggregate’s row lock, as in the SQL above. Keep the event id as well, for tracing, for a dead-letter record and for a second consumer that prefers it. The version is the outbox equivalent of the source log position that the rest of this site orders by: a number the source assigns in commit order that a replay carries unchanged.

Why not just use the log position of the outbox insert, as for any other table? Two reasons, one documented and one reasoned. Documented: the router does not forward the source block (above), so it is simply not on the message. Reasoned, from the PostgreSQL rule that concurrent transactions are decoded in commit order: the outbox row is only ever inserted, so no row lock orders two inserts for one aggregate. If transaction B inserts its outbox row, then transaction A updates the aggregate and inserts its row and commits, and only then B updates the aggregate and commits, the log shows A before B, and B’s outbox insert has the lower position. A guard that says “higher position wins” would discard B’s event. The version has no such problem because it is taken after the lock. This interleaving needs several connections, so I did not run it; a reviewer reproduced it on PostgreSQL 18 with three connections. Insert order and commit order differed. With test_decoding, B’s insert was at LSN 0/17C1868 and A’s at 0/17C19A8, so LSN order followed insert order, while the transactions were decoded in commit order, A’s first. That is the inversion described here: B’s event arrives second with the lower position. If you take the aggregate’s lock first in every transaction, the positions agree with commit order too, and you may read source.lsn from the raw change event instead of using the router (verify the position field for your connector, as described on Making sinks bulletproof).

The guard

One statement does both jobs. Apply an event only if its version is higher than the last one applied for that aggregate; a duplicate has an equal version and a stale event a lower one, so both are skipped. The delete is an event like any other and sets a marker, so a late older update cannot bring the row back (the resurrection problem).

Consumer: the guarded upsert (run on PGlite and SQLite)
CREATE TABLE shipments (
  order_id text    PRIMARY KEY,
  status   text,
  deleted  boolean NOT NULL DEFAULT false,
  last_seq bigint  NOT NULL
);

-- Apply one event: (order_id, status, deleted, seq). Zero rows changed means
-- a duplicate or a stale event.
INSERT INTO shipments (order_id, status, deleted, last_seq)
VALUES ('ORD-221', 'shipped', false, 2)
ON CONFLICT (order_id) DO UPDATE
   SET status   = excluded.status,
       deleted  = excluded.deleted,
       last_seq = excluded.last_seq
 WHERE shipments.last_seq < excluded.last_seq;
Four arrival sequences for one order, applied by an id-only dedupe and by the version guard (run on PGlite and SQLite)
Arrival (event seq) Id-only dedupe ends at Version guard ends at
1, 2, 3 then 2 again (duplicate) seq 3 seq 3
1, 3, 2 (swapped) seq 2 (stale) seq 3
1, 2, 3, then ids pruned, then 2 replayed seq 2 (stale) seq 3
1, 2, 3, delete at 4, then 3 replayed seq 3 if the id was pruned (row back); otherwise deleted deleted

Id-only dedupe passes the first row and fails the next two. That is the gap the public pages leave open: the router reference says the id can be used to remove duplicates, and the Debezium blog dedupes by id, and both are right about duplicates. Neither claims it orders events.

Pruning the seen-ids table

The Debezium blog suggests housekeeping on the consumer’s message log: periodically delete the events older than the consumer’s committed offset, because such messages will not be delivered to it again. That holds for the broker redelivering to the same consumer group. It does not hold for a repeat from the relay or connector, which, as the table above shows, arrives at a new, higher offset. Pruning by offset therefore does not bound how old a duplicate can be. Two safer rules:

  • Keep the last applied version per aggregate. It is one row per aggregate, it needs no pruning, and an event of any age is compared against it. This is the guard above.
  • If you must keep ids, prune by time, not offset, and keep them longer than the longest rewind you can perform: the topic retention, the slot’s WAL retention or the longest offset reset you would run, whichever is greatest.

Dense versions, with one condition. Because the version rolls back with the transaction, there are no gaps in an aggregate’s versions, so a consumer that sees 4 after 2 can conclude that event 3 is missing or late, park the event and alert rather than apply it. That holds only if every version bump writes exactly one outbox row. A repair script that runs UPDATE orders SET version = version + 1 without an outbox insert leaves a permanent gap, and a strict consumer then parks every later event for that aggregate forever. Route all changes through the same code path, or give the repair script an outbox insert. The check is the application’s to write; it is stricter than the SQL above, which accepts any higher version.

The id-reuse hole. The guard compares versions per id, so it breaks if an id is deleted and then created again with its version starting over. On PGlite, after versions 1 and 2 and a delete marker at version 3, applying the re-created aggregate’s first event (version 1) changed 0 rows: the new aggregate was silently dropped. Either never reuse an aggregate id, or add a generation (epoch) to the version and compare the pair, as below. A late event from the old generation then also loses (0 rows on PGlite), which is correct.

Consumer: the guard keyed on (epoch, version) (run on PGlite)
CREATE TABLE shipments (
  order_id text    PRIMARY KEY,
  status   text,
  deleted  boolean NOT NULL DEFAULT false,
  epoch    bigint  NOT NULL,
  last_seq bigint  NOT NULL
);

INSERT INTO shipments (order_id, status, deleted, epoch, last_seq)
VALUES ('ORD-221', 'created', false, 2, 1)
ON CONFLICT (order_id) DO UPDATE
   SET status   = excluded.status,
       deleted  = excluded.deleted,
       epoch    = excluded.epoch,
       last_seq = excluded.last_seq
 WHERE (shipments.epoch, shipments.last_seq) < (excluded.epoch, excluded.last_seq);

The producer must then write the epoch with each event (a column on the aggregate that increments on re-creation). Without a deliberate epoch, treat id reuse as a design error.

Outbox table growth and cleanup

An outbox table only gets inserts, so without a plan it grows forever. Which plan is safe depends on the relay.

When a physical DELETE of outbox rows is safe
Relay Safe to delete Why, and what it costs
Log tailing, delete in the same transaction Immediately The connector reads the insert and the delete from the log, not the table. The router discards the delete. The table stays empty and needs no housekeeping. Cost: nothing in the table to inspect, audit or re-send from (see below).
Log tailing, delete later by a job Any time after commit The same reason. Keep rows for as long as you want a record to look at. Delete with DELETE or by dropping a time partition; never flag rows with an UPDATE, which the router rejects.
Polling publisher After the row is marked published, plus a safety window The poller finds rows by reading the table, so a row it has not yet marked must stay. Delete rows published more than N days ago, with N longer than the time you need to investigate or re-send.

Insert and delete in one transaction. The Debezium blog does exactly this: persist the event row and remove it right away. It explains why that works: log-based CDC does not look at the table’s contents but tails the append-only transaction log, where the committed transaction contains both the insert and the delete. Debezium sends a message for the insert and ignores the delete, so the table is always empty, and “no separate house-keeping process is required”. The blog sets tombstones.on.delete=false so no deletion marker follows. The current reference confirms that the router filters out DELETE operations. The log still records both changes, and the log is trimmed on its own schedule.

PostgreSQL: write the event and delete it in the same transaction (run on PGlite)
BEGIN;
UPDATE orders SET status = 'delivered', version = version + 1 WHERE id = 'ORD-221';
INSERT INTO outbox (id, aggregatetype, aggregateid, seq, type, payload)
SELECT '6f1d0c1e-3b1f-4a52-9d0e-5b7f1a2c3d4e', 'order', id, version,
       'OrderDelivered', jsonb_build_object('status', status)
  FROM orders WHERE id = 'ORD-221';
DELETE FROM outbox WHERE id = '6f1d0c1e-3b1f-4a52-9d0e-5b7f1a2c3d4e';
COMMIT;

What you give up (my reasoning, not from the sources). If the outbox table holds nothing, the log is the only copy of the events. If the connector’s slot is dropped, or invalidated because it fell too far behind, the events committed in the gap cannot be rebuilt from the outbox: a snapshot of an empty table recovers nothing, and you must reconstruct from the business tables. Keeping rows cuts both ways. A few days of rows, or a partition you drop later, buys a record to compare against the topic and something a snapshot can re-send. But a snapshot or incremental snapshot of a table that still holds rows publishes every retained row again (the router passes snapshot reads through), with the same id and seq at new offsets. The consumer guard absorbs that; an id-only dedupe with pruned ids does not. PostgreSQL’s documentation notes that dropping or detaching a partition is much faster than a bulk DELETE and avoids its VACUUM overhead. The slot risk is covered on Postgres replication slots and WAL growth.

This is not the delete-marker problem. Deleting outbox rows is housekeeping on a queue. The rule on Deletes that stay deleted (keep a marker with a position) is about the consumer’s state, the shipments row above. And a topic’s retention, not the outbox table, decides how long a new consumer can replay; the blog makes the same point.

Ordering per aggregate in Kafka

The router uses the aggregateid column as the message key. Kafka writes events with the same key to the same partition and guarantees that a consumer reads a partition’s events in the order they were written. That is the whole ordering guarantee: per key, in one partition. Across aggregates there is no order.

  • Put events that must be ordered under one aggregate id and one aggregate type. The type picks the topic by default, and two topics have no order between them. The Debezium blog says the events of a child entity (an order line) should carry the root’s type and id for this reason.
  • Adding partitions to a topic changes the key-to-partition mapping for new messages. The Kafka operations documentation warns that this can disturb consumers who rely on it and that Kafka does not redistribute existing data. During that change the version guard is what keeps a consumer correct.
  • Producer retries need idempotence or one in-flight request to keep order. Kafka documents that with enable.idempotence true ordering is preserved for up to five in-flight requests per connection. Your Kafka Connect worker’s producer settings and defaults depend on your version: check.
  • A polling publisher has to preserve the order itself (section above), because it chooses the send order.

For how keys, skew and partition counts interact outside the outbox, see Partitioning and the partition key entry.

Testing the outbox path

Assert on the consumer’s state after a fault, not on how many messages were sent: a pipeline that sends one message twice and ends correct is behaving as designed. The method and a reusable fixture are on Testing a CDC pipeline; these are the outbox-shaped cases to add to it.

Faults to inject and what the consumer state must be afterwards
Fault How to inject it Expected end state
Relay crash between send and record Publish a batch, kill the relay or connector before the position or the published_at flag is saved, restart. Same rows as a clean run; the duplicate was skipped. See crash and restart.
Same batch twice Feed the consumer the same events two times. Unchanged after the second pass. See duplicates.
Swapped events for one aggregate Deliver seq 3 before seq 2. The state of seq 3, and seq 2 skipped. See out of order.
Replay after the seen-ids table was pruned Prune by your retention rule, then re-publish old events at new offsets. Unchanged. This is the case an id-only design fails.
Late update after a delete Deliver the aggregate’s delete, then an older update. Still deleted. See delete, then a late older update.
Update to an outbox row Run an UPDATE on one outbox row in a test database. The router’s table.op.invalid.behavior outcome (warn, error or stop) is the one you chose.

Where exactly-once stops

Say it per hop. Application to database: one transaction, so the state change and the event row commit together or not at all. That is the outbox’s real guarantee. Database to Kafka: the relay is at-least-once, as above. Debezium’s documentation (from 3.3) describes an opt-in exactly-once mode for source connectors on Kafka Connect; it covers that one hop, and the same page says it is unclear whether the underlying transaction implementation is fully correct. See Is CDC Exactly-Once?. Kafka to the consumer’s own store: a separate system with no shared transaction. What you can claim is exactly-once processing: duplicates and reordering happen, and the version-guarded write makes them harmless. See Exactly-Once Semantics.

Try it

Open the Change Feed Playground and pick the Outbox Relay scenario (its id is outbox-relay). It runs eight changes: an orders insert, three orders updates, three inserts into an outbox_events table with ids EVT-221-1 to EVT-221-3, and a delete of EVT-221-1. Look at two things. The ids end in a counter per order, which is the kind of embedded version this page recommends; a UUID would carry none. And the delete of EVT-221-1 appears in the log as an ordinary delete event, which is exactly the event Debezium’s router would drop. The scenario does not run a relay, repeat an event or dedupe, and each outbox insert is a separate change from its order update rather than one transaction. So the useful exercise is on paper: write down the event sequence a consumer would see if the connector restarted after EVT-221-2, and what your guard does with each.

Transactional Outbox Knowledge Check

Test whether you can tell what the outbox guarantees from what the consumer still has to do.

Q1

A service commits its database transaction, then calls the Kafka producer, and crashes between the two. What does the outbox change?

Q2

Outbox event ids are random UUIDs. A consumer must apply the events of one order in order. What should the events carry?

Q3

With a log-tailing relay, the application inserts the outbox row and deletes it in the same transaction. What reaches Kafka for the delete?

Q4

A consumer prunes its seen-event-id table of every id older than its committed Kafka offset. The connector is rewound and re-publishes old outbox events. What can happen?

0/4 correct

What to do Monday

  1. Find every place a service changes its database and then sends a message. Each is a dual write until the message is an outbox row in the same transaction.
  2. Add a per-aggregate version and increment it with UPDATE ... RETURNING in the same transaction that writes the outbox row. Do not read it first and add one in the application.
  3. Decide the relay. If it is Debezium, capture the outbox table only, apply the router only to it, and write down which version of the reference you read.
  4. Put the aggregate id in the key and the version where the consumer can read it (a header or the envelope), and check what a plain consumer actually receives.
  5. Make every consumer apply an event only if its version is higher than the last one applied for that aggregate, and keep deletes as markers.
  6. If a consumer keeps seen event ids, prune them by time, longer than your longest rewind, never by offset.
  7. Choose the cleanup: delete in the same transaction (log tailing only), a partition you drop, or delete after publish plus a safety window. Never UPDATE outbox rows when the router is in use.
  8. Run the six faults in the testing table against staging, and keep the failing case you find as a regression test.
  9. Mark each “check your version” item above with an owner and a test.

What was run, and what was not

  • Read on 2026-10-09: the Debezium Outbox Event Router reference (stable pages, which are 3.7, and the 1.9, 2.7, 3.0 and 3.3 versions for option names), the Debezium blog post on the outbox pattern, the Debezium PostgreSQL connector page (“Behavior when things go wrong”), the Debezium exactly-once page, microservices.io (the transactional outbox, polling publisher and transaction log tailing pages), the PostgreSQL pages on logical decoding, output plugins, transaction isolation and partitioning, and the Kafka introduction, producer configuration and operations pages. The router’s apply() method was read in the 3.0.0.Final and 3.4.0.Final sources (the class moved in later releases and was not read there).
  • Run: the producer SQL (the two transactions, a rolled-back transaction, a global sequence consumed by a rollback, the polling cycle with a repeated SELECT, the insert-then-delete transaction, and the id-reuse case with its (epoch, version) guard) on PGlite, which embeds PostgreSQL. The consumer guard and the four arrival sequences in the table ran on PGlite and on SQLite; the SQLite run is part of this repository’s tests.
  • Not run: Debezium, Kafka Connect and Kafka, so the connector configuration, the routed message layout (including a header arriving as bytes) and the router’s behaviour on an update or on a tombstone are from the documentation and the 3.0.0.Final source only. Concurrent database sessions: PGlite has one, so I did not run the lock-wait behaviour or the interleavings. A reviewer reproduced the commit-order inversion (with test_decoding) and the polling gap on PostgreSQL 18 with three connections; the lock wait is from the PostgreSQL documentation. The effect of an immediately-deleted outbox on slot loss is my reasoning. No managed database service was used.
  • Check your version: router option names and defaults, the partition placement, whether tombstones are dropped, producer defaults in your Kafka Connect worker, and the isolation level your application uses.

Further resources

On this site

Primary documentation (read 2026-10-09)

Progress 0% No progress yet
Progress is stored locally in this browser.