A fraud rule can be correct about an identifier and wrong about the person behind it. If one phone appears under several device IDs, a rule counting distinct devices can overstate the number of devices involved. If several phones share an identifier, the same rule can miss activity.

Before connecting a device signal to a blocking decision, make its behavior measurable.

1. Separate device changes from identifier changes

Incognia's May 2026 discussion of signal quality distinguishes a customer changing phones from an existing phone being assigned a new identifier. Both can appear as a new device in downstream systems.

Make those separate cases in your evaluation. Record which physical test device generated each event, the returned identifier, and the relevant software change. Compare an unchanged device, a legitimate replacement device, and a change in browser or app environment. A label such as new_device should have a documented meaning.

2. Test the assumptions an attacker can change

The device-obfuscation explainer describes how reinstalls, resets, emulators, and cloned applications can interfere with recognition. An identifier is only useful for repeat-abuse detection if you understand when continuity breaks.

In a controlled environment you own, define the expected result for each relevant scenario before running it. Record both recognition and integrity signals. An unsupported or manipulated environment should be distinguishable from an ordinary customer returning. Do not treat a passed recognition check as proof that every other reported attribute is trustworthy.

3. Inspect what happens to the account graph

Incognia's network-graph article explains why identifier continuity affects relationships between accounts and devices. When one device fragments into several apparent devices, connected activity can look unrelated.

Test the relationship your rule needs: whether several test accounts on one device stay connected, and whether separate devices stay separate. Inspect the underlying events as well as the visualization. A graph can look convincing while its identity assumptions are wrong.

4. Measure the intervention, too

The smart-friction article describes escalating verification according to risk. Detection and intervention therefore need separate evaluation.

For a limited pilot, compare the proposed decision with known test outcomes. Track unnecessary challenges, legitimate completion after a challenge, missed abuse, and review workload. Keep ordinary device upgrades and software changes in the legitimate test set. A stronger-looking rule is not an improvement if it mainly sends returning customers through repeated verification.

A compact evaluation record

Keep the signal version with the results so later changes can be compared. The useful deliverable is evidence about when a signal supports a decision, where it fails, and what the user experiences when it does.

Source author credits: Danny Paulk (device obfuscation and smart friction), Alice Lucena (network graphs), and André Ferraz (signal quality). This article adapts their existing Incognia publications; it does not attribute the new wording or proposed evaluation plan to them.