Distributed transactions: Reference architecture

Domain mutations and outbox rows share one database transaction; a relay publishes at-least-once; consumers are idempotent; writing the database and a broker independently without an outbox is forbidden.

Package status: reference context ready for human review. The contract and test scenarios are complete, but no claim is made that an adopting implementation has passed them.

Decision

When a domain mutation must notify other services, the event row is inserted in the same transaction as the mutation. A publisher relay reads unpublished outbox rows and delivers them to the broker at-least-once. Consumers treat duplicates as already processed using an event or deduplication key. Application code must not write the database and then independently produce to a broker without this outbox (or an equivalent transactional inbox on the other side).

Scope

• Transactional outbox table, relay polling or log tailing, at-least-once publish, consumer idempotency, and poisoning of undeliverable payloads.
• Backend aggregates that emit domain events.
• Coordination with job-queue visibility for relay workers.

Outside this block

• Event payload schema evolution rules.
• Webhook signing toward third parties after the event is already in the broker.

Contract

• Commit of the aggregate change is atomic with insert of one or more outbox records for that change.
• Outbox records include event_id, aggregate_id, occurred_at, payload, and publish_state of pending, published, or failed.
• The relay marks published only after the broker acknowledges; crashes before ack retry the same event_id.
• Consumers ignore duplicate event_id (or equivalent idempotency key) after the first successful apply.
• Dual-write without an outbox row in the mutation transaction is a policy violation detected in review and in an optional static check.
• Poison messages that fail decode after a budget move to failed with an operator path; they do not block the entire relay forever.

Implementation

• Insert the outbox row on the same connection as the aggregate mutation.
• Relay in small batches with skip-locked claiming; put schema identity on the payload.

Failure handling

• If the broker is down, pending rows accumulate and mutations still commit.
• Only acknowledged ids become published; oversized payloads fail that row only.

Verification

• Crash after commit and before ack; restart must republish pending ids.
• A consumer seeing the same event_id twice applies once; dual-write without an outbox row is a defect.

The executable-looking examples in this package are fixtures and acceptance contracts. Run python tools/validate.py from the collection root to check package structure and metadata; then implement and execute the scenarios in the target repository.

Adoption assumptions

• Names and numeric values in example.yaml are an adoption profile, not universal defaults.
• The adopting team must map actors, data classes, error vocabulary, and ownership to its system.

References

• The transactional outbox pattern is described at https://microservices.io/patterns/data/transactional-outbox.html.
• Event envelopes may follow CloudEvents when the adopting team chooses that envelope; at-least-once plus consumer idempotency remain required regardless of envelope.