When Fraud Looks Like Error
For years, the federal government has reported that about 3 percent of unemployment insurance payments are lost to fraud. That number has always seemed too low, especially in the last ten years, as stolen identity information has proliferated. The fraud rings that exploited unemployment assistance during the pandemic didn’t just appear, they had already learned— in ordinary times— just how easy government programs were to exploit. Organized fraud rings had been buying stolen identities on deep and dark web forums for several years and the pandemic became an irresistible opportunity to monetize them.
A preliminary analysis we at the Program Integrity Alliance (PIA) conducted, led by PIA Senior Fellow Dr. John Deke, brings some important data-driven insights to payment accuracy measurements in unemployment assistance. We analyzed data collected through the Benefit Accuracy Measurement program, or BAM, which is the tool the Department of Labor uses to measure accuracy in unemployment assistance payments. Our new analysis suggest that official fraud rate the Department reports is, in all likelihood, structurally too low.
What BAM is, and What it Misses
BAM is one of the strongest fraud measurement tools the federal government has. Unlike most fraud data, which comes from cases that were referred, investigated, and adjudicated, BAM starts by drawing random samples of paid unemployment assistance claims and investigating them. Because the sample is random, we know what spending the estimates cover, which lets us express fraud loss as a share of the program (a $1 billion loss means something very different in a $10 billion program than in a $100 billion one). We can also calculate standard errors, which tell us how much the estimate would bounce around from one random sample to the next.
But a random sample won’t guarantee the right answer. BAM investigations are conducted under real-world constraints, including limited resources to collect all information necessary to detect fraud, inconsistent classification rules, and time pressure. And BAM’s job is not specifically to find fraud, it’s to assess payment accuracy. Whether a payment was wrong matters; precisely why it was wrong is far harder to determine.
As a result, BAM produces two separate figures: an overpayment rate and the fraud rate, which is the subset of overpayments determined to be fraud. The fact that BAM publishes a fraud rate at all is laudable, even if the numbers are less reliable. Most agencies don’t try to estimate fraud, they simply use the broad term overpayments and let outside analysts decide how much of that might be fraud. The question our analysis of BAM data asked is whether some of what gets labeled “overpayment” is actually fraud that investigators couldn’t confirm.
The Identity Theft Signal
To investigate that question, knowing the outsized role identity theft plays in modern unemployment fraud, we assembled a decade of state-level BAM data and linked it to the Federal Trade Commission’s measure of identity theft reports per capita. The finding was unexpectedly stark. As it turned out, FTC identity theft reports are strongly predictive of the total BAM overpayment rate. More specifically, they are strongly predictive of overpayments minus what BAM classifies as fraud. In fact, about 90 percent of the relationship between identity theft and overpayments sits in the portion of payments that BAM does not label as fraud.
That is a significant result. It means that when identity theft rises in a state, total overpayments rise, but BAM-classified fraud doesn’t move with it. The identity theft signal is not showing up in the fraud column. The most straightforward interpretation is that identity-theft-related fraud is being absorbed into the overpayment category, where investigators are finding something wrong but not finding enough evidence to call it fraud. BAM’s classification rules for fraud require establishing willful misrepresentation. But this may be an untenably high bar for classifying payments, given that the purpose of this accuracy tool isn’t to build a case for court, but instead to estimate the magnitude of likely fraud.
At Least 27 Percent More Estimated Fraud
To translate this into a number, we asked a deliberately conservative question: how little would BAM fraud need to increase to make it behave the way BAM overpayments do when exposed to the identity theft signal? The answer, in non-pandemic years, is that the BAM fraud rate needs to rise by at least 0.91 percentage points— from 3.31 percent to 4.22 percent. That is a 27 percent increase in the measured fraud rate, and it is explicitly a floor. It captures only the identity-theft-related fraud that the FTC signal can detect. It says nothing about identity theft-base fraud that doesn’t generate an FTC report, or about a baseline level of under-detection that applies uniformly across all states and years. Those sources of undercount are real, but we don’t have the data to measure them yet.
To get a sense of how much the overpayment rate might rise with more resources devoted to investigation, we looked at what happens when other programs apply stronger measurement. When SNAP applies federal validation to state estimates, measured overpayments rise modestly—about 6 to 9 percent. When the IRS uses detection-controlled modeling to estimate unreported tax liabilities, the gap between what’s measured and what’s real is nearly double. When Medicare applies independent intensive re-review to sampled claims, measured improper payments more than double, and sometimes quadruple. These aren’t direct estimates of BAM’s under-detection because the programs are different, and the constructs aren’t identical. But they establish a principled range: the true fraud rate in UI is almost certainly higher than 4.22 percent, and depending on how much ordinary BAM investigations miss, it could be substantially higher.
The analysis doesn’t directly measure how the composition of UI fraud has changed over time, but the circumstantial case is strong. A decade ago, UI fraud was primarily an individual enterprise—someone recently laid off who kept certifying they were looking for work when they weren’t, or a claimant who went back to work and didn’t report it. That kind of fraud is bounded by the actual unemployment assistance population and limited to marginal misrepresentation.
Today, the dominant threat is systemic identity theft-based fraud. Data breaches have become daily occurrences, and they have handed organized criminal networks the raw material to file claims at industrial scale against identities that belong to people who have no idea their information is being used. The pandemic proved the model worked, and criminals spread the word. Counter to what some believe, the fraud didn’t end when the pandemic did, we just stopped looking as hard.
The threat vector matters because BAM’s classification rules were designed for the older fraud type. Establishing willful misrepresentation is straightforward when you’re looking at a real claimant who kept certifying incorrectly, but nearly impossible when the “claimant” is a stolen identity, the perpetrator is overseas, and the genuine identity holder can’t be located to corroborate what happened. If the composition of unemployment assistance fraud has shifted toward organized identity theft — and the evidence strongly suggests it has — then BAM’s structural undercount is an inherent measurement problem.
Undercounting Fraud Has Policy Implications
The deeper problem this analysis surfaces goes beyond unemployment insurance. Under the Payment Integrity Information Act and OMB guidance, agencies are required to estimate and report improper payments, including fraud. But those estimates depend on the quality of the measurement instrument the agency uses. If the instrument systematically misclassifies fraud as overpayment, the agency will dutifully report a low fraud rate that is technically consistent with its data and genuinely wrong.
BAM is one of the better measurement tools in the federal government. It uses random sampling, has well-defined statistical properties, and has been in use for decades. And even it systematically understates fraud. Agencies using weaker measurement tools like administrative case data, self-reported figures, and detection-driven samples almost certainly have a larger gap between what they report and what is actually happening.
This affects policy directly. Fraud estimates drive resource allocation, program design decisions, and legislative priorities. If the estimates are structurally too low, the urgency of the problem is systematically underappreciated.
The analysis also clarifies why pre-disbursement identity and eligibility verification matters so much. BAM is missing fraud because identity theft is difficult to detect after payment, especially when the genuine identity holder doesn’t know their information was used and can’t be reached to help unravel the claim. Pre-disbursement verification— checking identity and eligibility before the payment goes out, using authoritative government records, etc.— uncovers fraud at a point where the signal is much cleaner and easier to identify. The question being asked in pre-disbursement verification is whether the claimed identity and circumstances match what the government’s own records show, which can be checked in real time.
The findings in our analysis are suggestive, not conclusive. The FTC identity theft measure is a proxy, which means it reflects reporting behavior as well as actual fraud, and reporting can vary across states for reasons unrelated to fraud prevalence. As a result of that variation, our measure of identity theft is a combination of true identity theft and random fluctuations in reporting, diluting the relationship between reported identity theft and fraud. The relationships the analysis identifies are correlational; the analysis can’t prove that rising identity theft causes overpayments, only that the two move together in ways that are difficult to explain otherwise. And the external benchmarks from IRS and Medicare are analogues, not direct estimates of how much BAM is missing.
What we can say with confidence is that evidence is now consistent with, and makes substantially more plausible, the hypothesis that official unemployment assistance fraud estimates are materially too low — and that the gap reflects a systematic feature of how fraud gets classified when detection is incomplete. The 4.22 percent figure is a defensible floor, but the ceiling is unknown, and building the tools to find out should be a priority.
Photo by Marija Zaric on Unsplash. Article first posted on GovIntegrity.