Managed model platforms look like the easy part of an AI inventory. Nothing is hidden. Every call is authenticated, billed, attributed to a role and recorded somewhere. Compared with finding a coding agent on a laptop, enumerating what runs in Bedrock, Vertex or Foundry should be an afternoon's work.
It largely is, and that is the trap. The enumeration is easy and the record is thin. On all three platforms, what gets written by default is the fact of the call: which model, which principal, when, how many tokens. What was actually sent to the model, and what came back, is a separate setting that somebody has to turn on. On AWS the documentation says so in four words: model invocation logging is disabled by default.
This guide covers the third discovery surface, after the endpoint and the tool server. What the control plane records without being asked, what you have to enable per platform, the three defaults that catch governance teams in particular, and why the default record falls short of what the EU AI Act asks a high risk system to produce.
The surface that is not a shadow problem
Most AI discovery work is about finding things that were never declared. The managed platform surface is different in a way that changes the whole approach.
Nothing here is undeclared. Every invocation carries an authenticated identity, lands in a billed account, and generates a control plane event. There is no equivalent of a local model producing no traffic, or a tool server installed by one developer in an afternoon. If a team is using Bedrock, somebody provisioned it, and the account is in your organisation.
So the question is not "what exists". It is "what was recorded, and is that enough to answer the questions we will be asked". Three of those questions come up constantly, and the default configuration answers only the first.
- Which models are in use, by whom, and how much? The control plane answers this by default on every platform.
- What data went to them? Not recorded by default anywhere.
- What did the system actually do, step by step, when something went wrong? Partially recorded at best, and for agents frequently not at all.
What the control plane records by default
AWS documents this more explicitly than the others, which makes it the clearest illustration of a pattern that holds across all three.
CloudTrail records management events by default, and Bedrock logs InvokeModel, InvokeModelWithResponseStream, Converse and ConverseStream as management events. So a Bedrock call appears in CloudTrail with no configuration at all. The documented example entry is worth reading closely, because it shows exactly where the record stops.
The shape of a default record. AWS's own example of an InvokeModel event carries requestParameters of {"modelId": "stability.stable-diffusion-xl-v0"} and responseElements of null, alongside the IAM user, the source IP, the user agent and the timestamp. That is a complete answer to who called which model when, and it contains nothing whatsoever about the prompt or the output.
This is a sensible default. Prompts and completions are high volume and frequently contain personal or confidential data, and a platform that logged them automatically would create a liability its customers did not ask for. The problem is not the default. It is that governance programmes routinely assume the record is richer than it is, and discover otherwise during an incident.
What you have to switch on, per platform
| Platform | Recorded without configuration | Content logging | Destination | Notable limit |
|---|---|---|---|---|
| Amazon Bedrock | CloudTrail management events for the main invocation APIs: model, principal, time, token counts | Model invocation logging, which the docs state is "disabled by default". Once on, it collects the full request data, response data and metadata | CloudWatch Logs, Amazon S3, or both. Same account and Region only | Bodies over 100 KB and binary data go to S3 as separate objects. Only the bedrock-runtime endpoint is covered |
| Google Vertex AI | Cloud audit records of API activity | Request-response logging, enabled explicitly through the SDK or a REST config call. Logs samples of requests and responses rather than all of them | A BigQuery table | Pairs larger than the BigQuery 10 MB row limit are not recorded. For fine tuned models the config sits on the endpoint |
| Microsoft Foundry | Azure resource and access activity, with governance applied through Entra identity, role based access control, content filters, network isolation and Azure Policy | Tracing and evaluation of agents and models, with built in monitoring dashboards that the documentation marks as preview | Azure monitoring surfaces | Foundry is the current name for what was Azure AI Studio and Azure AI Foundry, and the resource model changed with it, so older runbooks and queries need revisiting |
Two observations worth carrying into a design review. First, every platform puts content logging behind an explicit decision, which means the decision has an owner and a date and should be recorded as a control. Second, the destinations are ordinary data stores you already govern: S3 buckets, CloudWatch log groups, BigQuery tables, Azure workspaces. Once content logging is on, prompt data inherits whatever retention, access control and residency rules those stores carry, which is a data protection review rather than an AI one.
Three defaults that catch governance teams
These are specific, documented and easy to miss, and each one breaks a governance assumption rather than a technical one.
-
Agent invocations are data events, and data events are not logged by defaultAWS documents that all Agents for Amazon Bedrock Runtime operations, including
InvokeAgentandInvokeInlineAgent, are recorded as data events, which CloudTrail "doesn't log by default". Capturing them requires advanced event selectors on the agent alias or inline agent resource types, and data events carry additional charges. The practical consequence is sharp: an organisation can have complete CloudTrail coverage of model calls and no record at all of its agents running, which is the opposite of the risk ordering most governance programmes would choose. -
Guardrail evaluations are data events too
ApplyGuardrail, including guardrail evaluations made during model invocation, needs its own selector. Until that is configured, the record of your safety control doing its job does not exist. A guardrail you cannot evidence is a control you cannot claim, and this is a one line configuration difference. -
Logging scope is chosen by modality, not by systemBedrock invocation logging is configured by selecting modalities: text, image, embedding, video. The documentation notes that selecting a modality logs for all models supporting it as input or output. So the scope of your content logging is a property of the account configuration rather than of the system you meant to govern, and a new workload inherits whatever was chosen months ago by someone else.
A measurement caveat in the same documentation. Streaming invocations produce an extra CloudTrail event carrying token counts, with an eventType of AwsServiceEvent. AWS warns that it "isn't an API call", "isn't authorized, and it isn't evaluated by IAM policies or service control policies", and that counting both it and the original call "overstates your call volume". If your AI usage metrics come from a CloudTrail query, check whether they are filtered to AwsApiCall before anyone reports them to a board.
There is a second attribution caveat worth knowing before anyone builds a control report on guardrail data. Where more than one guardrail evaluates a request, AWS states plainly that "the order of entries isn't guaranteed, and the CloudTrail event doesn't identify which guardrail produced each assessment", and advises against using position to attribute an assessment. A report that attributes blocks to specific guardrails from this data is inventing the attribution.
Why the default record does not meet Article 12
None of this would matter much if the obligations only asked for usage metrics. They ask for more.
Article 12 of the EU AI Act requires high risk systems to be built for "the automatic recording of events (logs) over the lifetime of the system", with traceability appropriate to the intended purpose, covering events relevant to identifying situations where the system may present a risk, to post market monitoring under Article 72, and to operational monitoring under Article 26(5).
Set the default cloud record against that standard and the gap is specific rather than vague. Knowing that a role invoked a model two hundred times last Tuesday supports none of those three purposes. Identifying a risk situation requires knowing what the system was asked and what it answered. Post market monitoring requires being able to characterise behaviour over time, not call volume. Operational monitoring requires enough detail for the person exercising oversight to see what they are overseeing.
Article 19 then sets the retention floor at "at least six months" for logs "under their control", with the same duty falling on deployers under Article 26. Both halves of that phrase matter in a cloud context. Platform log retention defaults are frequently thirty days, which is below the floor. And "under their control" is a question with a different answer depending on whose account the workload runs in. Our guide to audit ready evidence covers the retention duties in full.
Building the cloud half of the inventory
The useful output is not a list of models. It is a set of answers that each tell you something you can act on.
-
Enumerate accounts and projects before modelsThe unit that hides things is the account, subscription or project, not the model. An organisation with a tidy production account and nine forgotten sandbox accounts has a discovery problem that no amount of model listing will reveal. Start from the billing hierarchy, which is complete by construction.
-
Ask which principal calls each model, not which model is enabledModel access being enabled tells you nothing. The useful query groups invocations by the calling identity, because that maps to a team, a workload and an owner, which is what the register needs. Bedrock's invocation log captures the caller ARN automatically for exactly this purpose.
-
Record the content logging decision per account, with a date and an ownerWhether content logging is on is a governance control, not a platform setting. Write down what is enabled, which modalities, where it lands, for how long, and who decided. This is one of the cheapest pieces of real evidence available and almost nobody keeps it.
-
Check agent and guardrail coverage separatelyBecause these sit behind data event selectors rather than default logging, they need their own check. The question to ask the platform team is not "do we have CloudTrail" but "do we have selectors configured for agent and guardrail resource types, and since when".
-
Reconcile against spend and against the registerCloud AI consumption is one of the clearest signals of a workload that exists and was never declared, and it arrives monthly whether or not anyone runs a scan. Our guide to AI spend as a governance signal covers the reconciliation.
-
Set retention deliberately, per systemTake the platform default as a starting point rather than an answer. Where the Act applies, six months is the floor and intended purpose sets the real period. Reconcile that with data protection retention rather than letting the shortest default decide.
The trade-offs nobody puts on the slide
Content logging creates a sensitive data store. Turning it on means prompts and completions, which routinely contain personal and confidential data, now sit in a bucket or a table. That store needs access control, retention, residency and a lawful basis. The governance improvement is real and so is the new exposure, and both belong in the same decision.
Volume and cost are not trivial. Data events carry additional charges on AWS, and full request and response bodies at scale are a meaningful storage line. Sampling, as Vertex does by design, reduces both and weakens any claim to complete traceability. That is a legitimate trade, and the choice should be recorded rather than inherited.
Platform coverage has edges. Bedrock invocation logging covers the bedrock-runtime endpoint and the docs are explicit that calls through other endpoints are not currently captured. Every platform has boundaries like this. Read them before claiming coverage, because the claim is what an auditor will test.
The platform is only one surface. A complete picture needs the endpoint, the tool servers and the applications as well. Cloud discovery produces confident, well attributed data about the portion of AI use that happens to run on a managed platform, and says nothing about the rest. Confident partial coverage is a more dangerous reporting position than acknowledged ignorance.
Frequently asked questions
Where this leaves you
The cloud surface is the one place in AI governance where the data is clean, attributed and complete, and that is exactly why the gap in it is easy to miss. The record is genuinely good at the question it answers. It just answers a narrower question than anyone assumes, and the wider answer is one configuration away in each platform, guarded by a sensible default that nobody is going to change for you.
The specific things worth checking this week are small. Whether content logging is on, and where it lands. Whether agent invocations are being recorded at all. Whether guardrail evaluations are. Whether the retention period clears six months. Whether anyone wrote down who decided each of those. None of it takes long, and all of it is the difference between a cloud inventory that enumerates and one that evidences.
Where a governance platform fits. Each cloud answers questions about itself, in its own schema, over its own retention window. What no platform produces is the layer that holds every system across all of them with an owner, a risk classification, the obligations it carries and the decisions made about its logging. Keeping that current across clouds, and producing it when someone asks, is what a platform such as Holistic AI is built to carry. The cloud tells you what ran. The register is what makes it accountable.
