Playbook

Duplicate payment investigation workflow

Separate duplicate webhooks, duplicate chain observations, and duplicate commerce intents with evidence-first triage.

01

Objective

Determine whether duplicate signals are benign retries, duplicate commerce intents, or reconciliation errors—before double fulfillment or double posting.

02

Prerequisites

  • Idempotency keys logged with payment_id and event type.
  • Commerce reference uniqueness rules documented.
  • Duplicate taxonomy distinguishes webhook vs chain vs order duplication.

03

Operational signals

  • Same payment_id with multiple Confirmed transitions.
  • Two payment_ids for one order reference.
  • Idempotency store shows skipped duplicates rising.

04

Decision points

  • Refund versus credit versus apply to balance.
  • Whether to halt fulfillment pending investigation.
  • Provider dispute versus internal matcher bug.

05

Escalation paths

  • Support → finance for customer balance impact.
  • Engineering → payment operations for handler bugs.

06

Failure modes

  • Issuing refund and fulfillment for both signals without investigation.
  • Deleting idempotency records to unblock handler.

07

Recovery patterns

  1. Collect provider event timeline for all involved payment_ids.
  2. Map commerce references across three planes.
  3. Apply resolution code from exception taxonomy.
  • Retries are normal. Webhook delivery is at-least-once. Design consumers to tolerate duplicates and out-of-order arrivals where possible.
  • Asynchronous by design. Payers, chains, and your servers operate on different clocks. UI and finance should not assume synchronous finality.
  • Eventual consistency. API reads, webhooks, and portal views may briefly diverge during transitions. Reconciliation jobs exist to converge truth.

Walkthroughs: /operations