AI Governance in the Public Sector

A citizen refused a benefit by an algorithm cannot take their business elsewhere. That single asymmetry explains most of what is distinctive about public sector AI governance. In a commercial setting, a bad automated decision is a customer service problem with a competitive remedy. In a public setting it is an exercise of state power, and the questions that follow are about due process, legitimacy and the right to contest, not only about accuracy.

Public bodies also operate under an unusual constraint: their AI is often the most consequential and the least well resourced. The systems that decide benefits, allocate inspections, or triage housing applications tend to sit in organisations without a machine learning team.

United States federal: what actually applies now

This is the part where a lot of published material is out of date, so it is worth being precise. On 3 April 2025, OMB issued two memoranda that rescinded and replaced the previous framework:

The most consequential change for anyone maintaining an internal control mapping is terminological. M-24-10 split covered systems into rights-impacting and safety-impacting categories. M-25-21 consolidates these into a single high-impact AI category: uses that materially affect rights, safety, access to government services, or significant agency operations. If your control library, your policy, or your training deck still uses the older pair of terms, it is mapped to a rescinded memorandum.

The obligations that follow high-impact classification are recognisable minimum practices: pre-deployment testing, an AI impact assessment, ongoing monitoring, transparency, and human oversight, applied before deployment and continuously afterwards. The enforcement mechanism is unusually direct. Agencies report their minimum practices for high-impact AI by 22 September 2026, and a use case that does not meet them must be discontinued. That is a stronger consequence than most private-sector governance regimes carry.

Two dates worth holding. Agencies had to designate a Chief AI Officer by 30 June 2025, and CFO Act agencies had to publish AI strategies within 180 days of issuance, meaning by 30 September 2025.

The Chief AI Officer as a borrowable template

The federal CAIO requirement produced something useful for everyone: a publicly documented version of a role most organisations are still improvising. The remit combines convening authority across a large organisation, ownership of the classification process that determines which uses get heavy scrutiny, responsibility for the inventory, and a route to stop non-compliant use.

The interesting design choice is that it is a single named accountable officer rather than a committee. Committees dilute accountability, and in a distributed organisation someone has to be answerable for a classification decision. Private-sector readers building the equivalent role can borrow the shape; we cover the surrounding structure in the AI governance team.

Europe: the fundamental rights impact assessment

The EU takes a different route to a similar end. Under Article 27 of the AI Act, deployers that are bodies governed by public law, and private entities providing public services, must perform a fundamental rights impact assessment before putting a high-risk AI system into use. The duty also reaches any deployer, public or private, of high-risk systems under Annex III points 5(b) and 5(c), which cover creditworthiness and credit scoring, and risk assessment and pricing in life and health insurance.

A FRIA is not a data protection impact assessment with different headings. Two features distinguish it.

It is organised by right. The assessment works through the fundamental rights potentially affected rather than through data flows. Dignity, non-discrimination, privacy, effective remedy, good administration.

Impacts cannot be netted off. A negative impact on one right may not be offset by a positive impact on another. Each right is assessed independently. This matters because efficiency arguments frequently take exactly the netting form: the system is faster overall, so a subgroup's worse experience is acceptable. Article 27 does not permit that trade to be made silently.

A workable FRIA structure:

The EU AI Act guide sets out the wider high-risk regime this sits inside, and our risk classifier helps establish whether a system falls into it.

Transparency registers

Several jurisdictions have concluded that the public should be able to see which algorithms their government uses. The UK maintains the Algorithmic Transparency Recording Standard, a defined format for publishing records about algorithmic tools in public sector decision making. Canada's Directive on Automated Decision-Making pairs a mandatory Algorithmic Impact Assessment with impact levels that scale the requirements, and publishes the results.

The honest case for publication is not that citizens read registers. Most do not. It is that the act of preparing an entry for publication changes internal behaviour. A team that knows a description will be public writes a clearer description, and questions it cannot answer surface before deployment rather than after a journalist asks. Publication is a forcing function first and a transparency measure second.

The cost is real and worth planning for: someone has to maintain the register, entries go stale, and a register that is visibly out of date is worse than none because it invites the inference that governance is performative.

Procurement is where the control actually is

Public bodies build less AI than they buy, which means the strongest governance point is the contract rather than the code review. By the time a system is deployed, the training data, the model architecture, the explanation capability and the change process have all been decided by someone else.

M-25-22 is the current federal expression of this, attaching requirements to solicitations issued on or after 30 September 2025 and to renewals or extensions exercised on or after 1 October 2025. Note the renewal trigger: it reaches existing vendor relationships at their next decision point, not only new procurements.

The clauses that matter most in a public context, beyond the general set covered in third-party and AI vendor risk assessment:

• Explanation capability as a requirement. If a decision must be explainable to the person affected, that is a functional specification, not a nice-to-have.
• Data rights and portability. Public bodies change suppliers on political and budget cycles. Lock-in is a governance risk, not only a commercial one.
• Right to test on your own population. Written in before signature, because afterwards it is a favour.
• Notice of material model change. With a period long enough to reassess before the change reaches the public.
• Transparency cooperation. If you must publish a register entry, the vendor must supply what the entry requires.

Contestability

This is where public sector governance departs most sharply from commercial practice. An affected person needs four things, and each has an operational cost:

• Notice that an automated system was involved, given at a point where it is still useful.
• An explanation in language they can act on, which means reasons rather than feature importances.
• Human review by someone with authority to reach a different conclusion and the information to do so.
• An appeal route with a timeframe.

The failure mode is a review that is formally available and practically unusable: a reviewer with two minutes per case, a screen showing the model's score and little else, and a workload that makes agreement the path of least resistance. That is the automation bias problem in administrative form, and it is a design choice rather than an inevitability.

The inventory problem, in a federated organisation

Both regimes assume you know what you have. Public bodies are frequently federated to a degree that makes this genuinely hard: agencies with their own IT, local authorities with their own procurement, arm's length bodies with their own budgets.

What works is to make the register the price of something people want. Access to a central AI platform, security review sign-off, procurement approval above a threshold, or budget release: attach registration to a gate the team already has to pass. Voluntary self-reporting alone reliably undercounts. The techniques in how to build an AI model inventory apply, with the added consideration that in a public body the inventory may itself become publishable.

What private-sector readers should take from this

Three things transfer well. The single named accountable officer, because it forces classification decisions to have an owner. The FRIA discipline of assessing each right separately rather than netting harms against benefits, which is a useful check on efficiency arguments anywhere. And the practice of writing a description intended for outside eyes, because it exposes what you cannot yet explain.

One thing does not transfer: the discontinuation consequence. Very few commercial governance regimes will actually switch off a working system for failing a control. Public ones increasingly say they will, and that changes how seriously classification is taken.