The Accountability That Exists on Paper But Not in Practice
The woman asks why. The officer is kind, even apologetic, but her answer lands like a door closing: “The AI system assessed your application and determined you do not meet eligibility criteria. I can tell you what the system decided, but I cannot tell you why it decided that in your specific case. The reasoning is not accessible to me either.”
This was an engagement between a woman who has just been denied unemployment benefits and a welfare officer
The necessary safeguards already exist: an appeals mechanism, parliamentary oversight, ministerial responsibility, and freedom of information laws.
Every layer of democratic accountability is in place, functioning exactly as designed. Except the thing those mechanisms were designed to hold accountable has quietly moved somewhere they cannot reach.
How Democratic Accountability Was Built to Work
The whole architecture of government accountability rests on a foundational assumption so basic it is almost never stated out loud: somewhere in the chain of any consequential public decision, a human being exercised judgment. That judgment can be examined, questioned, defended, overturned, or punished.
Ministerial responsibility, judicial review, parliamentary scrutiny, ombudsman processes, administrative law. The chain runs from the affected citizen, upward through the decision-maker, to the oversight body, and finally to the minister who is responsible. It is a chain designed to carry weight.
Until very recently, nobody questioned this assumption because it could not be otherwise. Decisions required human judgment. Technology assisted humans, but the moment of judgment was always human.
When the Judgment Disappears
From the assessment of benefits eligibility and the assignment of risk scores to the allocation of resources and the shaping of licensing decisions, every step is now driven by algorithms and machine analysis.
In each of these contexts, a citizen’s outcome is substantially determined by a non-human process.
The accountability chain does not break so much as become incoherent. The minister is responsible for policy.
The civil servant is responsible for implementation, vendor responsible for system performance, but nobody seems responsible for the specific decision in the way accountability frameworks require.
When a court asks what judgment was exercised and whether it was lawful, the honest answer is that the judgment was produced by an algorithm whose reasoning cannot be fully articulated. The review mechanism has nothing to grip.
What makes this particularly difficult is that the performance of accountability continues even as the substance retreats.
Ministers still answer parliamentary questions, officials still respond to information requests and reviews still happen. But the thing these processes were designed to reach is no longer where it was expected to be.
How Political Choices Become Infrastructure Decisions
There is a question that should be asked loudly and publicly before any AI system takes over a consequential government function: how much decision-making authority should we centralise in automated systems, and what transparency do citizens deserve when those systems affect them?
These political questions involve trade-offs between efficiency and accountability, between speed and scrutiny, between operational convenience and democratic principle.
Instead, these questions tend to arrive dressed as infrastructure decisions. “We are implementing a new benefits assessment system.” Not: “We are shifting consequential judgments about citizen entitlements from caseworkers to algorithms.”
The first framing goes through procurement and IT governance. The second would require parliamentary debate, public consultation, possibly legislative authority.
The technical frame is not dishonest in intent, It only shows how these decisions genuinely look from inside government. But the effect is to remove from democratic contest some of the most consequential choices a government can make about how it exercises power over citizens.
The Incentive Structure That Keeps the Problem in Place
Programme leaders are rewarded for delivery, speed, and efficiency. Finance teams are rewarded for cost savings and headcount reduction. Communications teams are rewarded for innovation messaging and digital transformation narrative. AI centralisation serves all of these goals well.
Raising accountability concerns, on the other hand, slows programmes and creates difficult conversations.
The immediate costs of raising concerns are high and visible. The long-term costs of not raising them are borne by citizens and by public trust.
Independent oversight bodies, parliamentary committees, and civil society organisations are the parties who should be raising the structural problem.
But most of them operate with mandates and resources designed for an earlier governance era, when the human at the centre of a decision was more straightforwardly identifiable.
They are trying to perform accountability in a context that has quietly changed underneath them.
What Restoring Accountability Coherence Actually Requires
Three requirements would actually restore coherence to accountability: interpretability, auditability, and liability clarity. Each is demanding, and none is optional if democratic accountability is to mean anything in an AI-assisted government.
Interpretability means the ability to explain a specific decision. Not “the AI scored you below threshold” but “the AI weighted these specific factors from your application in these specific ways, producing this score.”
Auditability means a record of how the system behaved across populations; one that independent oversight can scrutinise to determine whether decisions are consistent with policy intent and treat demographic groups equitably.
Liability clarity means a defined chain of human responsibility when the system produces a consequential error.
Who is accountable: the official who approved deployment, the vendor who built the system, the minister whose policy it implements? That question cannot remain ambiguous.
These are not features to be added after a system is running. They have to be requirements written into procurement before a contract is signed.
Building accountability into systems already deployed and operationally embedded is an order of magnitude harder than building it from the start.
Most government AI systems currently operating were deployed without these requirements. Fixing that retrospectively is genuinely difficult, which is exactly why the procurement stage is the leverage point that matters.
The Accountability Failures Government Should Be Preparing For
A citizen sues government over an AI-determined benefit denial. The court asks government to explain the reasoning behind the specific decision. Government cannot provide an explanation detailed enough to meet judicial review standards, because the system was not built to produce one.
The court rules that the decision-making process violates the administrative law requirement for transparent, examinable judgment. The entire system is suspended.
An independent audit reveals that an AI system systematically disadvantages a specific demographic group.
Government cannot explain why, because the system reasoning was not auditable. The media coverage, the public outcry, and the political fallout are severe.
Trust in government digital services collapses across departments that had nothing to do with the original system.
A parliamentary committee investigating government AI use asks for evidence that systems make decisions consistent with policy intent.
Government cannot provide that evidence. The inquiry concludes that government deployed systems without adequate accountability safeguards.
The full accountability crisis has not arrived yet, largely because government AI deployment is still relatively limited in scope.
As centralisation accelerates, the probability of an accountability failure that cannot be managed quietly increases.
Two Paths, One Decision Point
Most government agencies are currently following a risky path: they use AI to speed up work and centralize power without first making sure the system can be held responsible for its mistakes. They keep up the appearance of oversight using old methods, but they are essentially waiting for a major crisis to reveal that no one is actually in control.
This isn’t a secret plot; it’s just what happens when people don’t make a different choice.
The better alternative is much harder. It requires making accountability a mandatory rule before any AI is launched. This means accepting that things will move slower and cost more at the start. It involves building systems that humans can actually understand and fix, ensuring that our ability to govern the technology grows as fast as the technology itself.
These life-changing decisions are happening right now in boring office meetings and quiet contract signings. Most citizens have no idea these technical choices are being made.
Every time a government buys an AI system without strict oversight rules, they are choosing the dangerous path. The more we deploy these “unaccountable” systems, the harder it becomes to go back and fix them later.
The Briefing Senior Officials Rarely Get
The comfortable narrative inside most government institutions sounds like this: “We are modernising services with AI, compliant with data protection and have oversight mechanisms in place. Accountability is intact.”
Each of those statements may be technically true, and together they add up to something deeply misleading.
Accountability is intact as a performance. Whether it is intact as a substance is a different and more uncomfortable question.
The officials who should be asking it are often the same officials whose incentive structures reward them for not asking it.
The woman denied unemployment benefits earlier did not lose her benefits because anyone wanted to treat her unfairly.
She lost them because a procurement decision made years earlier, by people focused on efficiency and modernisation, did not include a requirement to explain the system’s reasoning in terms she or a court could examine.
The accountability gap between her and that decision is not a technology problem. It is a governance problem. And it is one that cannot be solved after the fact.
The next AI procurement that crosses a senior official’s desk carries a question embedded in its pages: does this system include interpretability requirements, auditability mechanisms, and clear liability chains?
If it does not, approving it is a choice; not an oversight, not a neutral technical decision, but a choice to deploy a system that democratic accountability was not built to reach.
That choice has consequences. It is just that those consequences tend to arrive later, and for someone else.
