Module 12 — Idempotency and Duplicate Prevention
This is one of the most important integration concepts in the whole bootcamp because distributed systems regularly retry, redeliver, and lose responses.
The key question is:
If the same logical request is processed twice, do I get one correct business result or two unintended side effects?
12.1What idempotency meansA simple definition:12.2Why duplicates happen even when your code is “correct”Imagine:12.3codeThe easiest Salesforce idempotency mechanism: External ID + UpsertSuppose source sends:12.4Business key vs Salesforce IDSuppose Mule gets:12.5External ID needs to be truly stableBad external ID candidate:12.6codeIdempotency keySometimes the operation itself doesn't naturally have a stable entity key.12.7Business key vs request idempotency keyThese solve different problems.12.8codeExample: order creationRequest:12.9codeExample where Upsert can be wrongSuppose source sends:12.10Where do you store processed idempotency keys?Common choices:12.11Mule Object Store mental modelMule's Object Store is effectively persistent key/value storage available for state such as:12.12But “check then insert” has a race conditionSuppose two Mule workers receive the same request simultaneously.12.13Atomicity mattersYou need some mechanism that guarantees:12.14Why Salesforce unique External IDs are valuableSuppose two Mule workers both simultaneously perform:12.15The Query → Create raceClassic bug:12.16codeIdempotency and database uniquenessIf Mule owns a database table:12.17Status matters tooSuppose request key exists:12.18codeA robust idempotency recordConceptually:12.19Reused key with different payloadFirst request:12.20TTL / expiryDo you store idempotency keys forever?12.21codeEvent deduplicationMessage:12.22But sometimes event ID dedupe is unnecessaryIf event means:12.23Idempotent update vs non-idempotent incrementThis is a useful distinction.12.24VersioningNow suppose events can arrive out of order.12.25Idempotency vs orderingMemorize:12.26Optimistic concurrencyThis is related.12.27Last-write-winsThis is the simplest policy:12.28Duplicate prevention at multiple layersA mature design may have:12.29Example: customer upsertRequirement:12.30Example: payment requestRequirement:12.31The hardest payment failureSequence:12.32Downstream-supported idempotency is bestIf payment API itself supports:12.33Salesforce Create vs Upsert under concurrent retriesBad:12.34Duplicate rules in SalesforceSalesforce itself can have duplicate-management rules.12.35Why email-based duplicate rules aren't enoughTwo Contacts:12.36Batch idempotencyNightly file:12.37Job-level idempotencySuppose same input file is uploaded twice.12.38Multi-level idempotencyA sophisticated import might have:12.39Idempotency and side effectsSuppose flow does:12.40Idempotent Salesforce write + non-idempotent notificationPossible fix:12.41Another classic problem: create then publishRetry after timeout:12.42Compensation is not idempotencySuppose duplicate Account is accidentally created and you later delete it.12.43Exactly-once business outcomeInterviewers may say:12.44Race condition scenarioInterviewer:12.45Retry scenarioSalesforce times out during Account creation. What do you do?12.46Duplicate event scenarioKafka/queue delivers the same customer event twice.12.47Out-of-order scenarioVersion 42 is processed, then version 41 arrives.12.48How to test idempotencyThis should be part of your automation suite.12.49Test 2 — repeated retrySend:12.50Test 3 — ambiguous timeoutMock downstream so that:12.51Test 4 — same idempotency key, different payloadRequest 1:12.52Test 5 — concurrent duplicate messagesPublish:12.53Test 6 — stale versionCurrent Salesforce:12.54Test 7 — workflow side effectsRequest does:12.55Common bad patternsunder concurrency.12.56Design checklistWhenever you see a write operation, ask:•cheat sheetModule 12 Cheat SheetThe interview sentence to memorize is: