The Dutch tax office turned a non-Dutch passport into a fraud score

The model that graded every claim

The Netherlands pays parents an allowance that covers most of the cost of childcare. It is paid in advance on an estimate, then reconciled against what the family actually earned and used. A low-income household can have up to 96 percent of its childcare bill covered, so the sums are large, and any gap between the advance and the final entitlement has to be paid back.¹ In 2013 the Tax Administration, which runs the scheme, added software to police it: a risk classification model that scored every incoming application for how likely it was to be wrong.¹

The model was self-learning, meaning it was trained on past applications marked correct or incorrect and adjusted its own weights over time, so the more a new claim resembled an old one that had been marked wrong, the higher its score.¹ A claim it scored low was approved automatically. A claim it scored high went to a civil servant, who could open an investigation and, if the paperwork did not satisfy him, label the applicant a fraudster.¹ Almost everything that went wrong is already visible in that arrangement.

How a passport became a risk factor

Among the model's several dozen inputs was one yes/no field: did the applicant hold Dutch citizenship?¹ Not being Dutch counted toward a higher score. This is what a proxy is, a fact that is not the thing you care about but stands in for it. The office was not measuring fraud. It was measuring resemblance to past flagged cases, and it had decided that a non-Dutch nationality was one of the resemblances worth points. A non-Dutch parent and a Dutch parent who filed identical claims did not receive identical scores, and the non-Dutch parent was likelier to be pulled out for investigation.²

The same reflex ran outside the model too. When officials suspected fraud among a small group of people with Ghanaian nationality, between 120 and 150 of them, they queried the database and investigated all 6,047 applicants who held Ghanaian nationality.¹ The suspicion of a few became a search of everyone who shared their passport. In July 2020 the Dutch Data Protection Authority established that nationality had been a factor in the model, and in December 2021 it fined the Tax Administration 2.75 million euros. In the regulator's own words, the office "used applicants' nationality (Dutch/not Dutch) as an indicator in a system that automatically designated certain applications as risky."²

A flag with no reason attached

The civil servant received a flag and little else. The model scored an application high-risk but did not say why, and the people who built it could not say either, because a self-learning model keeps no plain-language reason for any single score.¹ The official had a flag, a family, a caseload, and an employer that rewarded finding fraud. Mostly, the official trusted the flag.

That reflex has a name: automation bias, the tendency of the human who is supposed to check a machine to defer to it instead, because questioning a number no one can see is harder than acting on it. The review that was meant to protect people became a stamp on a score no one could inspect. The family was boxed in the same way. They were told they were under investigation but not told what the software had counted against them, so they could not argue with it.¹ The opacity was not a side effect. A score no one can see is a score no one can contest, and a system no one can contest has thrown away the safeguard that makes it survivable to be wrong.

Why a false alarm meant ruin

Two more facts turn a wrong flag into a catastrophe. The first is that real benefit fraud is rare. Hunt a rare thing with a rough test, and most of what the test catches is not the thing. Flag hard on weak signals and the result is mostly false alarms.¹ The office had a reason to flag hard anyway: its fraud unit was expected to claw back enough money to cover the cost of running itself, an incentive to seize whether or not the accusation held.³

The second is the rule waiting for a flagged family. It was all or nothing. A missing signature on a childcare contract, or a personal contribution paid a little late, was enough for the office to reclaim every euro of benefit it had ever paid and cut off all future payments, the whole sum due at once.¹ Because the benefit was largest for the poorest families, the poorest families owed the most. Any parent recorded as owing more than 3,000 euros, or as having received more than 10,000 in a year, was automatically marked as having committed deliberate fraud.¹ No one checked the mark, and it stripped away the right to repay in instalments. A paperwork slip became a five-figure debt due tomorrow.

The wreckage is documented. Tens of thousands of families were driven into debt, unemployment, eviction, and divorce, and the parliamentary inquiry that examined it in 2020 called the injustice "unprecedented."³ Statistics Netherlands later estimated that about 2,090 children of affected families were placed in out-of-home care.⁴ A government commission tied that harm to the debt and stress the scandal created rather than to any decision by the tax office itself, a distinction the piece keeps: the placements were court-ordered on child-safety grounds.⁵ The scandal, public since 2018, brought down the government. The third Rutte cabinet resigned on 15 January 2021.⁶

The algorithm did not act alone

It is tempting to say an AI did this. That is the wrong lesson, and the record does not support it. Take the strongest case for blaming the model first: it used nationality as a proxy, so it steered non-Dutch families into the investigation pipeline to begin with, and its opacity kept them there.¹ Against that, the journalist Jesse Frederik, who reconstructed the affair, argued there were tens of thousands of victims but no single culprit. The ruin came from harsh law, a rule with no partial setting, and a culture of distrust that had brushed off warnings for years. It would have fallen on whoever was selected, by the model or by a manual sweep.⁷ That culture was not confined to one program: the same office kept a separate secret blacklist of suspected fraudsters with no legal basis, which drew its own record fine.⁸

The regulator's finding splits the difference. It established that nationality was in the model and its use unlawful, but it did not establish that officials cut off benefits because of nationality once a case was open.² So read the failure as three switches rather than one. The nationality proxy chose who entered the punitive process. The black-box opacity disabled the human brake inside it. The all-or-nothing rule set how completely a false alarm destroyed a family. Pull any one switch and the harm shrinks. What would settle how much each mattered is per-family data comparing flagged and unflagged families with the same errors. That data is mostly missing, because the system was built in a way that never produced it.

Only the nationality proxy has actually been addressed. The fine punished it. A Dutch court had already struck down a sibling system, SyRI, a welfare-fraud scoring tool aimed at poor neighbourhoods, as an unlawful intrusion in February 2020.⁹ But the compensation scheme for the benefits victims excludes redress for the model's discrimination, and nothing in the record undoes the all-or-nothing logic for the people it caught.¹ The shape is portable. Any system that scores citizens for fraud on a proxy, hides the reason, and hands the score to an official under pressure inside a punitive rule will fail the same way, whatever it is called.