First diagnose where duplicates come from
A duplicate does not, by itself, justify a new CRM. Separate existing dirty data, native matching configuration and an integration that repeats writes. Pause unsafe automatic merges, keep an export and test with fictional records before changing live data.
| Observed symptom | Check first | Proportionate next step |
|---|---|---|
| Old duplicates remain; no new ones appear | Import history and field mapping | Reversible cleanup with a merge log; preserve relationships and preferences. |
| Typing the same email creates a new contact | Native duplicate rules in the actual plan | Configure and test the existing CRM before building a connector. |
| Each webhook retry creates a record | Event ID, idempotency and retry logs | Repair the integration; replay an event without duplicating its effects. |
| Two people share a company inbox | Identity policy and additional verified identifiers | Manual review; email alone is insufficient. |
Two similar records do not always represent the same person. Two enquiries from the same person do not always represent the same work. Before automating CRM deduplication, separate two questions: who does this information belong to, and have we already processed this submission? Confusing them can discard a new opportunity or create two tasks for one request.
1. Separate identity from repetition
A submission identifier helps recognise the same event when it arrives again. A stable reference from the source system can help recognise a contact within that system. These are different keys and must retain their context. Do not use a name as a unique identifier or an isolated timestamp as proof that a message is new.
If a contact asks for another service, keep the new enquiry even when linking it to an existing record. If the form resends the same request during a retry, avoid creating another task. Define how long submission identifiers will be remembered and what happens to older repeats.
2. Define matches and exceptions
- Compatible stable source reference: a candidate for linking, after checking that it belongs to the same scope.
- Matching email: useful evidence, but inspect shared mailboxes and conflicting details before automating.
- Same name with a different email: human review; do not merge on resemblance.
- Shared, recycled or incomplete phone number: insufficient evidence of identity on its own.
Preserve the original value and document any normalisation. Trimming surrounding spaces from a validated field may be reasonable; removing dots or “+” tags is not a universal rule. RFC 5321 requires preserving case in the mailbox local part: do not lowercase every email assuming it always identifies the same address. Policy must fit the system and its data.
3. Decide what to preserve before merging
A match does not authorise overwriting the whole record. Decide which source is authoritative for each field and what happens when sources disagree. Preserve ownership, history and provenance. A contact restriction should not disappear because a new form arrives; receiving an enquiry does not imply permission for campaigns. Uncertain data needs to remain visible to someone authorised to resolve it.
Fictional example: Alex Example has alex@example.test and an account owner. A new enquiry from that address may be linked while keeping the owner. Another from alex.other@example.test with the same name goes to review. No enquiry is deleted and records are not merged automatically. The reviewer should see the reason for the match and record their decision.
4. Recover failures without repeating effects
A timeout does not prove a write failed: the destination may have saved the contact before the response was lost. Check the state using a stable reference or use the provider’s documented idempotency mechanism. RFC 9110 explains HTTP method idempotency; it does not make every POST creation safe to repeat.
Agree retry limits, an exception owner and a visible pending state. Record the identifier, operation and outcome needed to investigate, avoiding unnecessary personal information in logs. Duplicate protection must also cover concurrent processes; searching and then creating, without a uniqueness guarantee or coordination, can introduce a race.
5. Test before using real data
- Repeated submission: a single task and a recoverable result.
- New request from a known contact: preserve the request and owner.
- Same name with a different email: review without automatic merging.
- Lost response after saving: inspect the destination before retrying.
- Insufficient permission or destination outage: visible exception without claiming success.
- Two simultaneous submissions: verify the agreed uniqueness rule.
Use invented data and inspect the destination result, not just the connector’s message. Prepare a recoverable copy before any real merge and check what each tool can undo. Rules, permissions and tests matter more than turning on a generic “remove duplicates” option.
Turn the policy into a workflow
Our local demo lets you explore a new enquiry, a known contact, ambiguity and a failure with retry. Recovery is simulated and does not validate a real provider. To discuss your case, describe the source, destination, owner and an exception using fictional data; we can then review the scope of a solution.
Try the enquiry and follow-up demo
Reproduce the decision with five inputs
Synthetic exercise, not an active provider integration. Use a stable event ID for delivery retries and a separate contact identifier. These IDs belong to one source and one workspace. A new enquiry is allowed even when its contact already exists; do not deduplicate distinct enquiries by email.
event_id | contact_id | email | enquiry
evt-01 | C-7 | ana@example.test | quote A
evt-01 | C-7 | ana@example.test | quote A
evt-02 | C-7 | ana@example.test | quote B
evt-03 | unknown | shared@example.test | quote C
evt-04 | unknown | missing | quote D| Input | Expected decision | Expected effect |
|---|---|---|
| 1: evt-01 | New event and known contact C-7 | One enquiry A attached to C-7. |
| 2: evt-01 again | Previously completed event | No new enquiry or task; return the recorded outcome. |
| 3: evt-02 | New event, same contact | Keep enquiry B separate from A; do not discard it. |
| 4: shared inbox | Ambiguous contact identity | Review queue; do not auto-merge people. |
| 5: missing identity | Insufficient data | Preserve the enquiry for review; do not guess or drop it. |
Acceptance: after all five inputs, there are two enquiries attached to C-7 and two items requiring review. Retrying evt-01 adds nothing. If a write succeeds but its acknowledgement is lost, reconcile the recorded event outcome before retrying. In a real connector, the event record and write must be atomic where possible, or use the provider’s supported idempotency key and reconciliation. A failed attempt must not be marked completed.
Limits before you merge real records
Never infer marketing consent from a duplicate match. Keep opt-outs, original IDs, ownership, relationships and the merge history. Do not lowercase the email local part, remove dots or strip plus tags universally; apply only rules verified for your providers. This exercise tests a policy, not an API or a production migration.
Try the local, synthetic enquiry demo
Ask us to review an existing configuration or integration
Compare integration with a new CRM only if the requirements justify it
Prepare the input, exception and acceptance check in a brief
Method and review
Reviewed by Taldrivo on 19 September 2026. This guide combines a synthetic worked example, our own software and the sources identified below. It does not claim customer outcomes, a provider price or compatibility with a plan that has not been checked.