The AI Reader.
When AI Breaks

Deloitte billed the Australian government A$439,142 for an assurance report citing books that don't exist

The fake citations came from a generative AI tool the report named only in its methodology, never on a cover page.

Why this matters

Assurance reviews exist so that someone can trust a system they cannot check for themselves. A government pays an outside firm to read the code and the records and say whether they hold up, and the firm's name on the finding is the whole reason the review is worth buying. In 2025 a Deloitte assurance report for an Australian government department cited a book that was never written, a court decision that never happened, and academic work its supposed authors never produced. The department had paid A$439,142 for it. By the end you will know how a generative AI tool produces a citation that looks real but points to nothing, and why an invented one is built exactly like a true one. You will also see where that same missing check sits in the reports and filings governments buy and publish every day.

Background optional reading
01
A model builds a fake citation the same way it builds a true one: the mechanism behind every fabricated reference, taken apart in full.
02
Steven Schwartz's ChatGPT brief cited six court cases that didn't exist: the same failure inside a court filing.
03
Meta's Galactica invented citations for real researchers: a model that fabricated references by design.
04
Robodebt built debts from averaged income, then made recipients disprove them: the welfare-compliance program the reviewed system sits beside.

The assurance the department paid for

In December 2024 the Australian Department of Employment and Workplace Relations paid Deloitte to check work the department could not vouch for on its own. It commissioned an independent assurance review of the Targeted Compliance Framework, the automated system the department uses to enforce welfare recipients' obligations.1 That framework sits in the same corner of welfare administration as robodebt, the debt-recovery program a royal commission later found unlawful. An assurance review is a paid, arm's-length examination, and its whole value is that someone outside the department reads the system and puts a firm's name to the finding.

The contract was worth A$439,142.00.2 Deloitte carried out the review between December 2024 and June 2025, dated its report 4 July 2025, and the department published it on 14 August.1

A book cited repeatedly that was never written

The footnotes did not survive a close reading. In late August 2025, Christopher Rudge, a lecturer in law at the University of Sydney's law school,3 went through the report and found that a number of its authorities did not exist. He counted close to twenty errors, among them roughly ten citations to a single book that was never written, and took what he found to the Australian Financial Review, which broke the story.4

Fabricated and misattributed authorities in the 4 July report, and what the 26 September correction did with each.56
In the 4 July report In the 26 September correction
The Rule of Law and Administrative Justice in the Welfare State, a book attributed to Lisa Burton Crawford, cited repeatedly through the footnotes and in the bibliography Removed. No such book exists.
A second Burton Crawford book, The Rule of Law and Administrative Discretion, at footnote 55 Removed. Also nonexistent.
A Carolyn Adams article in the UNSW Law Journal, at footnote 14 Replaced with a different, real Adams article.
A Björn Regnell software-engineering paper, at footnote 66 Removed entirely.
A speech by "Justice Natalie Cujes Perry", at footnote 58 Replaced with Justice Melissa Perry's real "iDecide" speech.
Amato v Commonwealth [2021] FCA 1019, at footnote 57, quoting "Justice Davis" Re-attributed to the real 2019 consent orders of Justice Davies.

Two of the people named are real, and their absent works are not. Lisa Burton Crawford is a professor of public and constitutional law at the University of Sydney Law School. She did not write the book cited repeatedly against her name.7 Björn Regnell is a professor of software engineering at Lund University in Sweden. The release-planning paper the report attributed to him was not his.8

Worse than any missing book was a court that never spoke. The report's treatment of IT-system defects turned on Amato v Commonwealth, a Federal Court matter from the robodebt line of litigation, which it cited as Amato v Commonwealth [2021] FCA 1019 and quoted directly.

The quotation the court never gave
Notably, her Honour Justice Davis stated at [25]-[26]: The burden rests on the decision-maker to be satisfied on the evidence that the debt is owed. A person's statutory entitlements cannot lawfully be reduced based on an assumption unsupported by evidence.5 The 4 July report, quoting a decision that does not exist

There is no 2021 decision. Amato was resolved in 2019, by consent orders that Justice Davies, not Davis, made in the Federal Court on 27 November. Victoria Legal Aid, which ran the case, records what the court's annexed statement actually said: that "there was no material before the decision-maker capable of supporting the conclusion that a debt had arisen under the Act and, therefore, the conclusion was irrational."9 The report's version paraphrases that gist and attaches it to a judge's name, a set of paragraph numbers, and a citation that were all wrong.

Real academics had their names on books and papers they never wrote. A Federal Court judge was quoted saying words she never said. All of it sat inside a document the Commonwealth had paid a global firm to get right.

Why a model invents a citation that looks real

How does a report cite a book that was never written? The answer was in the report itself, if a reader reached page 147. Deloitte disclosed, in two places, that its technical review had used a generative AI tool: an Azure OpenAI GPT-4o system, licensed by the department and run on the department's own servers. Neither disclosure appeared on a cover page or in a summary. One sat inside a methodology graphic on page 48. The other was in Appendix C, "Detailed Methodology: Technical Review", a note that the work "included the use of a generative artificial intelligence (AI) large language model (Azure OpenAI GPT-4o) based tool chain licensed by DEWR and hosted on DEWR's Azure tenancy."6

A model of this kind does not look a reference up and copy it down. When it produces a citation, it generates the most statistically likely run of words to follow the ones before, the same way it produces any other sentence. A real reference and an invented one come out of the identical process, so a fabricated citation carries every surface mark of a true one: a plausible author, a plausible title, a plausible year and page. Nothing in the model confirms that the work exists, because confirming is not part of what it does.

The same process has produced the same failure before: a science model that fabricated references by design (Meta's Galactica), and a New York lawyer who filed six invented cases in a real court (the Avianca brief). Why a model builds a fake citation exactly as it builds a true one is the subject of an earlier lesson. What is new here is the setting: not a demo and not one careless filing, but a paid deliverable a government commissioned, received, and published.

Deloitte corrected the report and refunded a fifth of the fee

On 26 September 2025 Deloitte issued a corrected report, which the department published on 3 October. A "Report Update" note on the second page recorded what changed: citations and reference-list entries "which contained errors", the summary of the Amato proceeding, and revisions "to improve clarity and readability". The same note stated that the updates "in no way impact or affect the substantive content, findings and recommendations in the Report".6 A department spokesperson likewise said only that "some footnotes and references were incorrect", with "no changes to the recommendations".10

Deloitte agreed to repay part of the fee. At a Senate estimates hearing in early October, officials confirmed the amount.

A$439,142the contract fee
A$97,587refunded, the final instalment

The refund was A$97,587.11, the contract's final instalment, or about a fifth of what the department had paid.11

Four things about the episode stay genuinely contested, and the record settles none of them. The first is scale. The department called the problem "a small number of corrections to references and footnotes";1 Rudge, who found them, put it near twenty errors, including the roughly ten citations to one non-existent book.4 Both can be literally true, since a small number of distinct fabricated works can be cited many times over.

The second is whether the correction was substantive. Deloitte's note insists the findings and recommendations did not move, and on the record they did not.6 Critics answer that fabricated legal authorities inside a legal-assurance report bear on whether the report can be trusted at all, whatever survived in the recommendations.4

The third is the refund. The government and Deloitte treat repayment of the final instalment as resolution. The Australian Greens, whose senator Barbara Pocock drew the figure out at the hearing, called it "measly" and demanded the full A$439,142 back.11 No one disputes the number. They dispute what it should have been.

The fourth is the cause. The disclosure and the pattern of the errors point to unverified model output entering the citations. Deloitte has published no detailed account of how the fabrications got in, so where exactly the human process failed is asserted by others rather than admitted in the document.6 The correction did not close the file either: on 3 February 2026 the department published a further-corrected version, replacing the September one.1

Where an unchecked citation gets in

What went wrong at Deloitte was a missing step, and the step is missing far beyond Deloitte. A generative AI tool helped produce a paid deliverable,6 and no one between the tool and the client opened a single citation to confirm the source was real. The same gap sits wherever these tools now draft: consulting reports, legal filings, and the submissions governments commission and publish. It reached journalism too, when CNET ran AI-drafted finance articles under a staff byline. The tool returns a fluent, correctly formatted reference on demand, and it reads exactly as trustworthy as a real one until someone checks. In each setting, the person who signs the document is the last check on whether its sources exist.

The takeaway

A generative AI tool builds a fake citation with the same confidence and the same formatting as a real one, because producing a citation was all it was doing in either case. Deloitte's report shows the cost when no one opens them: a book that does not exist, a Federal Court quotation the court never gave, and a refund of A$97,587.11 the government and its critics still cannot agree was enough. The tool made careless work read exactly like careful work, and the only thing that separates them is someone opening a source to see whether it is real. Anyone who commissions or signs AI-assisted work now owns that check.

Go deeper optional reading
01
AI Incident Database, incident 1193: a running record of the documents and coverage on the Deloitte report.
02
Deloitte's corrected report (archived PDF): read the Appendix C disclosure of the AI tool in full.

Sources

  1. Department of Employment and Workplace Relations · Targeted Compliance Framework assurance review, resource page
  2. AusTender · Contract notice CN4118426, Assurance Review to support the Targeted Compliance Framework
  3. The Conversation · Christopher Rudge, contributor profile
  4. The Nightly · Law lecturer Christopher Rudge slams Deloitte's government-funded report written with AI
  5. Deloitte for DEWR · Targeted Compliance Framework Independent Review, Final Report, 4 July 2025 version
  6. Deloitte for DEWR · Targeted Compliance Framework Independent Review, Final Report, 26 September 2025 corrected version
  7. AUSPUBLAW · Lisa Burton Crawford, contributor profile
  8. Lund University · Björn Regnell, faculty profile
  9. Victoria Legal Aid · A depth look at our robo-debt test case, Amato v Commonwealth
  10. Accounting Times · Deloitte to refund government after using AI in $440k report
  11. Australian Greens · Greens slam Deloitte's unethical behaviour and measly refund over AI report