After Deployment: Post-Market Monitoring and Model Drift

Almost everything an AI governance programme produces is a judgement about a moment. The classification describes the system as it was understood in March. The assessment records what testing found that week. The approval says a named person was satisfied on a date. All of it is accurate and all of it starts ageing immediately.

Post-market monitoring is the only control in the set whose job is to notice that something you already approved stopped being true. It is also the one most programmes never build, for a structural reason rather than a careless one: approval is a milestone with a date and an audience, and monitoring is an ongoing cost with no event attached to it until the day it finally fires.

This guide covers what the obligation actually asks for, what changes in a system when nobody in your organisation changes anything, the four layers worth measuring, why you cannot borrow a drift threshold from anyone, and how the loop connects back into assessment. It sits after our guide to AI incident response, which owns what happens once something fires, and this guide does not restate the reporting clock.

The only control that runs after the decision

Set out a governance programme's artefacts in order and the shape of the gap is obvious. Inventory, classification, assessment, testing, approval, deployment. Six steps, all of them producing a record that describes a state, and then nothing until an incident or a scheduled review a year later.

The assumption buried in that sequence is that a system which passed assessment continues to behave the way it behaved during assessment. For conventional software that assumption is usually sound, because the artefact is deterministic and the organisation controls when it changes. For AI systems it is unsound in three distinct ways, and only one of them is about the model.

• The world moves. The population the system sees stops matching the population it was validated on. Nothing about the system changed; its inputs did.
• The use moves. A system approved for one purpose gets pointed at an adjacent one that nobody reassessed, usually because it worked well enough.
• The system moves. For anything built on a hosted model, the component can be updated by its provider on a schedule you do not set and are not told about.

Monitoring is the only control that catches any of the three. An annual review catches them up to a year late, by which point the question is no longer whether to intervene but how long it has been wrong.

What changes when you change nothing

The third case is the one that breaks people's intuitions, so it is worth having a concrete measurement rather than an assertion.

In 2023, Lingjiao Chen, Matei Zaharia and James Zou measured the behaviour of hosted GPT models at two points three months apart and published the comparison as How is ChatGPT's behavior changing over time?. On one task, identifying prime versus composite numbers, they report GPT-4 as "reasonable" in March at 84% accuracy and "poor on these same questions" in June at 51%.

Three things about that study matter more than the headline number.

It is a preprint. No peer reviewed venue is listed, and anyone citing it in a governance paper should say so. It is the clearest public measurement of the effect, not a settled result.

The direction was not uniform. GPT-3.5 improved on the same task where GPT-4 declined. GPT-4 improved on multi-hop questions while GPT-3.5 got worse. Both produced more code formatting mistakes in June. This is the important part for anyone designing monitoring, because it rules out the comforting assumption that updates are improvements and that a single aggregate quality metric will catch a problem. A change that improves two tasks and breaks a third nets out to nothing in an average.

The mechanism was behavioural, not accuracy. The authors present evidence that instruction following decreased over time and identify it as a common factor behind the changes. A system whose prompt scaffolding assumed a particular obedience to formatting instructions can break without any underlying capability being lost.

What Article 72 actually asks for

Article 72 requires providers of high risk systems to "establish and document a post-market monitoring system" proportionate to the technology and the risks. Four features of the drafting shape what you have to build.

1. It covers the whole lifetime, actively
   The system must actively and systematically gather and analyse performance data over the system's whole lifetime. "Actively" rules out a passive arrangement where nothing happens unless a user complains, which is what most organisations have.
2. The test is continued compliance, not accuracy
   The purpose is to evaluate continued compliance with the Chapter III, Section 2 requirements. That is a wider bar than model performance: it reaches accuracy and robustness, but also human oversight, record keeping and transparency. A monitoring design that only watches a quality metric answers one part of the question.
3. Interaction with other AI systems is in scope where relevant
   An explicit requirement, and an unusual one. Where one system's output feeds another, the composition has behaviour neither component has alone, and this is the clause that says somebody has to look at it.
4. The plan is part of the technical documentation
   The monitoring system must be based on a post market monitoring plan which forms part of the Annex IV technical documentation. It is not a separate operational document that lives with the platform team. It sits inside the file an authority can ask for, which means it is written to be read by someone outside the team.

The template is the timing detail worth knowing. The Commission is required to adopt guidance including a template for the plan, with the Board's opinion given the utmost weight, by 2 September 2027. Anyone building now is building ahead of it. That argues for a plan whose structure is easy to remap rather than one entangled with a particular monitoring tool, and for keeping the plan's content separable from the dashboards that implement it.

There is also a sensible reuse provision. Providers of systems covered by Annex I Section A harmonisation legislation may integrate these elements into monitoring systems and plans that already exist under that law, and the same option applies to Annex III point 5 systems placed on the market by financial institutions subject to EU internal governance rules, provided the result gives an equivalent level of protection. If you already run post-market surveillance for a regulated product, the answer is usually to extend it rather than to build something parallel.

Four layers worth measuring

Monitoring designs tend to collapse into one layer, usually output quality, because that is the layer with the most available tooling. The failures that matter are spread across four.

The output layer deserves a specific technique, because it is the one that answers the hosted model problem. Keep a fixed probe set: a few dozen inputs with known expected behaviour, covering the cases your system actually depends on, including the formatting and instruction-following behaviours your scaffolding assumes. Run it on a schedule and on every version change. It costs very little, it is the only thing that detects a silent upstream change, and the Chen result is precisely what it would have caught.

The fourth layer is the one that distinguishes governance monitoring from machine learning monitoring, and it is almost always missing. Where a system is classified as advisory because a human reviews its output, the override rate is the measurement that tells you whether that classification is still true. Our guide to risk tiering covers why that number so often moves a system up a tier.

Why you cannot borrow a drift threshold

The natural next question is what counts as too much drift, and the honest answer is that the published evidence will not tell you.

The literature on behaviour change in deployed language models is thin, and the figures that circulate in vendor material do not hold up when traced. A frequently quoted degradation percentage turns out to come from a 2022 study of 128 model and dataset pairs that predates language models entirely. A 2026 preprint proposing an agent stability index derives its headline figure from simulation rather than field data. Another reports a quality drop from an internal experiment with no published method. None of those is a threshold you can defend to an auditor, and citing one is worse than citing nothing.

Thresholds then come from consequence rather than from statistics. A movement that would change who gets approved is material at a far smaller magnitude than one that changes how a summary is formatted. Setting thresholds by statistical significance rather than by consequence produces alerts on changes nobody needs to act on and silence on the ones that matter.

Building the loop

1. Capture the baseline at approval, not later
   The moment the system is signed off is the only point where you have an agreed definition of good. Record input distributions, output distributions, probe set results and the intended override rate then. Reconstructing a baseline six months in means baselining a system that may already have moved.
2. Build the probe set before you need it
   A few dozen inputs with known expected behaviour, covering the dependencies your application actually has. Version it, run it on a schedule and on every upstream version change, and keep the results. This is the cheapest high value component in the whole design.
3. Set the cadence from the tier, not from the tooling
   Monitoring everything continuously is expensive and produces alert fatigue that ends in the alerts being muted. A Tier 1 system that decides outcomes for people deserves continuous input and output monitoring with outcome review monthly. A Tier 3 system can be a quarterly probe run. Proportionality is in the Article and it is also what makes the programme survivable.
4. Decide in advance what each trigger does
   An alert with no predetermined response becomes a recurring agenda item. For each monitored signal, write down the threshold, who is notified, what they are authorised to do, and at what point the system is suspended. The deployer duty to suspend where a risk appears is much easier to meet when somebody decided beforehand what that looks like.
5. Route findings back into assessment, not into a report
   The loop only closes if a monitoring finding can trigger reassessment and reclassification. Where monitoring reports into an operations review and assessment lives in governance, findings accumulate in one place and change nothing in the other. Name the path and the person who walks it.
6. Keep the output as evidence, with a retention period
   Monitoring produces one of the few continuous evidence streams a governance programme has, and it is often held in a tool with a thirty day default. Our guide to audit ready evidence covers the retention duties this has to clear.

The trade-offs nobody puts on the slide

Monitoring finds things, and finding things costs money. A programme that starts monitoring properly will generate findings that need triage, investigation and sometimes remediation, on systems that were previously quiet. That is the control working and it is also a real operational cost that nobody budgeted. It is better raised before the first month's findings than after.

Outcome monitoring touches people's data. Measuring what happened downstream of a decision means tracking individuals through a process, which engages data protection and sometimes employment law. The layer with the most governance value is the layer with the heaviest privacy footprint, and designing it with the privacy team rather than presenting it to them afterwards is the difference between a six week and a six month conversation.

Alert fatigue is the normal failure mode. Monitoring programmes do not usually fail by missing a signal. They fail by producing so many that the recipients stop reading them, and the muting happens informally so nobody records that the control stopped working. Fewer signals with predetermined responses beat comprehensive coverage nobody reads.

You cannot monitor your way out of a bad baseline. If the system was never properly assessed, monitoring tells you it is behaving consistently with a state nobody validated. Drift detection on an unvalidated system detects change, not correctness, and the two get conflated in reporting.

Frequently asked questions

Where this leaves you

The reason post-market monitoring gets deferred is that it is the only governance control with no natural completion. Everything else finishes. The assessment is done, the approval is granted, the file is filed. Monitoring is a standing commitment that produces nothing visible for months and then, occasionally, produces the single most important finding the programme has.

What makes it tractable is accepting that it does not have to be comprehensive to be useful. A fixed probe set, an input distribution check and an override rate will catch all three ways a system moves away from the thing that was approved. The expensive version is not more accurate, it is more thorough, and thoroughness is what gets switched off when the alerts pile up. The version that survives is the one somebody still reads in month nine.