An AI Duplicate-Record Assistant That Knows When Not to Merge
Updated: Nov 12
Two records can describe the same customer. They can also describe two different people with the same name, two branches of a company or a shared family email address. That is why cleaning a database is not simply a matter of deleting anything that looks similar.
An AI duplicate-record assistant can help find candidates and explain the evidence. Its most valuable output is a review queue with clear reasons, not a large number of automatic merges. A clean-looking database is not useful if important relationships, history or privacy choices have been lost.

Start with the rules you already know
A record is a structured entry in a system, such as a customer profile in a customer relationship management system, usually called a CRM. Duplicate detection compares those entries to identify cases that may represent the same real-world person or organization.
Exact checks use agreed identifiers or precisely matching fields. Similarity checks look for likely variations, such as “Acme Limited” and “Acme Ltd.” AI can help interpret messy names and descriptions, but many useful checks need no language model at all.
Salesforce distinguishes matching rules, which identify potential duplicates, from duplicate rules, which determine how the system responds. This is a helpful design principle for any implementation: recognizing a possible match is separate from deciding to block, warn or merge. Salesforce duplicate-management guidance.
Define what counts as the same entity
Before testing a model, decide what the database is meant to represent. Should each company branch have its own account? Can one person legitimately have several roles? Are customer and supplier records intentionally separate?
Write down the fields that strongly identify an entity and the fields that merely suggest similarity. A verified internal customer identifier may be decisive within one system. A shared postcode or switchboard number is usually weaker evidence. Missing information should not be treated as matching information.
Normalization can make comparisons easier. It means creating a consistent comparison format, such as trimming extra spaces or standardizing phone-number formatting. Preserve the original values. Do not silently rewrite names or remove meaningful differences just to make matches easier.
An illustrative customer-record review
Imagine a fictional services company with two records for “Maya Ellis” at the same organization. One was created after an event and one from an inquiry form. The email addresses match, but the job titles and communication preferences differ.
A third record shares the same company telephone number but belongs to a different employee. The assistant should distinguish the stronger two-record candidate from this weaker coincidence.
A practical workflow is:
Read a limited, approved set of records and preserve their identifiers.
Run exact comparisons for trusted identifiers and agreed fields.
Generate additional candidates through name, address and other similarity checks.
Compare each candidate's evidence, conflicts and related records.
Present likely duplicates with an explanation and a suggested review priority.
Have an authorized data owner decide whether to keep separate, investigate or merge.
If a merge is approved, apply field-by-field retention rules and verify the resulting record.
AI should not decide that two people are the same solely because their names resemble each other. Its explanation must point to the actual evidence used.
What a review card might show
An illustrative card could read:
Candidate records: C-104 and C-287.
Shared evidence: Same email address and organization; names match after whitespace normalization.
Conflict: Different job titles. Neither has been independently confirmed as current.
Privacy warning: One record contains an email opt-out. Do not replace that choice with a more permissive value during cleanup.
Related activity: One record has an open support case; the other has a proposal.
Recommended action: Owner review before merging. Preserve both activity histories and resolve the title separately.
Another card might say, “Keep separate: different named contacts share the organization's main telephone number.” Preventing a false merge is a useful result, even though it does not reduce the record count.
Avoid presenting a similarity score as a probability of identity unless it has genuinely been calibrated for that purpose. “Strong candidate: same verified email and matching organization” is often more understandable than an unexplained “97% match.”
Merging has consequences beyond the visible fields
A customer profile can be linked to cases, campaigns, files, accounts and access settings. The merge screen may not show every consequence.
Salesforce's contact-merging documentation describes effects on privacy records, relationships, contact roles, portal users and related items. The exact behavior depends on the platform and record type. That is why a merge process needs testing in the actual system, not just a spreadsheet comparison. Salesforce merge considerations.
Before enabling writes, establish a recoverable backup or export strategy, understand the platform's recovery limits and test the process on non-production data where possible. Keep a log of the source records, retained fields, reviewer and final record identifier.
Do not assume that selecting the newest record preserves the best data. A recent import may be less reliable than an older verified entry. Retention rules should consider source quality, confirmation date and the meaning of each field. Privacy choices require particular care and should follow the organization's approved policy.
Fix the source of duplicates too
Cleaning existing records is only half the job. Review where duplicates originate: inconsistent imports, forms that always create new profiles or integrations that use different identifiers.
A prevention step could warn a staff member when an existing record looks relevant. Another could update a verified record rather than creating a new one, using a stable external identifier. Test legitimate exceptions so prevention does not block real customers with shared details.
For a pilot, take a limited export with appropriate access controls and have a data owner label known duplicate and non-duplicate pairs. Include difficult examples, not just obvious matches. Measure false positives, missed duplicates, review time and the number of cases correctly left separate.
Keep the first live version read-only. Once reviewers trust the evidence, consider narrowly scoped, approved merge workflows. A useful starting point with 48PEGASUS is identifying the system where duplicate records cause the most practical trouble, then designing a review process that protects the information people still need.





Comments