Inbound payment webhooks acknowledge receipt separately from ledger mutation, deduplicate by provider event identity, reconcile unknown outcomes via provider fetch, and treat the internal ledger as authoritative when ordering or payload conflicts arise.
Package status: AI-assisted reference package. Static package validation has passed; implementation scenarios remain not run and human domain review is required before stable adoption.
Decision
Payment provider webhooks arrive on an authenticated ingress endpoint that verifies signature before persistence. Invalid signatures return 401 without storing a receipt. Valid payloads persist as receipt records; HTTP 200 returns after durable storage, before ledger mutation. A fast dedupe cache may expire after dedupe_store_ttl_days, but a durable immutable event registry keeps provider event identity for at least the ledger and audit retention period. Manual_review resolves to applied or rejected through operator_resolution. Unknown outcomes remain reconciling until provider fetch succeeds or timeout moves them to manual_review.
Scope
- Webhook receipt acknowledgement, signature verification, deduplication store, reconciliation worker, provider fetch client, and ledger mutation idempotency.
- Manual review resolution paths and post-dedupe TTL behavior.
Outside this block
- Outbound webhook delivery to customers.
- Entitlement projection updates (see entitlements block).
- PCI card data collection UI and stored credential vaulting.
Contract
- HTTP 200 indicating receipt success is sent only after signature verification passes and raw payload is durably stored with provider event id.
- Invalid signature returns 401; invalid_signature_stores_receipt is false; no receipt row is created and no ledger mutation is scheduled.
- Duplicate provider event id within dedupe_store_ttl_days acknowledges success without re-applying ledger mutation.
- After the fast dedupe cache expires, the provider event id is checked against the durable event registry; a previously applied event acknowledges successfully without another ledger mutation.
- A genuinely unseen event may enter reconciliation only after its provider event id is inserted atomically into the durable registry or an equivalent uniqueness constraint wins.
- Ledger mutation never occurs in the HTTP handler thread; it runs in a worker that transitions reconciliation state independently of acknowledgement.
- manual_review transitions to applied or rejected only with operator_resolution audit records suitable for finance export.
- Provider fetch and webhook payload conflicts on amount block automatic credit and enter manual_review.
- Late-arriving events with older effective timestamps do not reverse settled ledger rows without an explicit reversal event type.
Implementation guidance
- Store raw webhook bytes before JSON parsing so signature verification can be replayed.
- Use reconciliation-states.yaml transitions in operator dashboards with separate metrics for acknowledgement versus ledger success.
- Schedule exponential backoff fetches for provider API errors; do not grant provisional access unless allow_provisional_access is explicitly true in the profile.
- Separate acknowledgement metrics from ledger mutation metrics to detect stuck workers after receipt.
Failure handling and safeguards
- Worker crash after acknowledgement resumes from stored receipt without double-ack to provider.
- Fetch timeout beyond profile limit moves to manual_review rather than defaulting to paid.
- Idempotency keys on ledger writes prevent duplicate charges when workers retry.
Verification and operations
- Tests cover invalid signature rejection, cache expiry with durable deduplication, manual_review resolution to applied and rejected, and out-of-order refunds.
- Metrics track acknowledgement-to-mutation lag, reconciling queue depth, manual review rate, and rejected terminal count.
- Reconciliation drills simulate provider outage during peak webhook volume.
Adoption assumptions
- Names and numeric values in example.yaml and reconciliation-states.yaml are an adoption profile, not universal defaults.
- The adopting team selects payment providers and defines ledger authority conflict rules with finance stakeholders.