A security team can show you every AI request that left the building last quarter. They can show you the gateway logs, the sanctioned model endpoints, the token spend by department. What they usually cannot show you is the coding agent running on a developer laptop that reads the whole repository, calls a model with a personal API key, and writes files back to disk without a single packet crossing the corporate proxy.
That gap is not a configuration mistake. It is the predictable result of where AI moved. For three years the assumption held that AI meant a request to a hosted model, and a request is something a network control can see, log and block. Desktop assistants, local models, editor agents and Model Context Protocol servers broke that assumption. The work now happens on the machine.
This guide is about the control layer that assumption left behind: the endpoint. It covers why network controls stopped being sufficient, what evidence actually sits on a device, what each layer of tooling can and cannot see, and how the gap connects to obligations you already carry under ISO/IEC 42001, the NIST AI Risk Management Framework and the EU AI Act. It is a companion to our guide on shadow AI, which covers the problem in general terms. This one is narrower and more technical: it is about the place where discovery usually fails.
Why the network stopped seeing your AI
Gateway based controls work on a simple premise: if every AI call is an outbound request to a known host, then inspecting outbound requests gives you complete coverage. That premise held when the only way to use a large model was to call somebody else's API.
Four shifts broke it, and all four happened quickly.
Desktop applications. An assistant installed as a native app manages its own connections. It does not necessarily respect system proxy settings, and where it uses certificate pinning, an inspecting proxy sees an encrypted tunnel rather than a prompt.
Coding agents. An agent embedded in an editor reads source, runs commands and edits files locally. The interesting security event is not the model call. It is the file access, the shell execution and the permission mode nobody reviewed.
Local models. A quantised model running on the device makes no outbound request at all. There is no traffic to inspect, because there is no traffic. Data governance questions about what was fed to it remain entirely unanswered by any network control.
Tool servers. Model Context Protocol servers and similar connectors give an assistant reach into ticketing systems, databases and cloud consoles. They often hold long lived credentials in plain configuration files, and they are installed by individuals rather than provisioned by IT.
The structural point. These are not edge cases that better network rules will catch. They are categories of AI use that produce no inspectable traffic by design. A control that works by watching requests cannot see work that never becomes a request.
What actually lives on the endpoint
The useful thing about local AI is that it is not actually hidden. It is unexamined. Tools that run on a machine leave ordinary artefacts on that machine, and most of them sit in predictable places because the tools were built for convenience rather than concealment.
None of this requires interception, keystroke capture or reading anybody's prompts. It is configuration and inventory data. That distinction matters a great deal when the works council or the privacy team asks what exactly you intend to collect, and it is the strongest argument for doing endpoint discovery by reading state rather than by watching behaviour.
What each control layer can and cannot see
Most organisations already own several of these. The question is rarely whether to buy something new, and usually whether the layers already deployed leave a gap worth closing.
| Layer | Sees well | Blind to |
|---|---|---|
| AI gateway or model proxy | Sanctioned model calls, prompt and response content, per team token spend | Anything not routed through it: native apps, local models, personal keys |
| CASB and secure service edge | Known SaaS AI destinations, uploads to web applications, unsanctioned sign ups | On device execution, pinned certificates, local inference |
| Data loss prevention | Sensitive strings in monitored channels | Intent and permission state, and anything in an inspection blind spot |
| Endpoint detection and response | Processes, binaries, malicious behaviour | AI meaning. A coding agent is a legitimate signed binary, so nothing fires |
| Device management and unified endpoint management | Installed software inventory, fleet wide policy push | Which software is AI, what it may access, and whether keys are exposed |
| Endpoint AI discovery | Local agents, tool servers, local models, exposed keys, permission state | Content of conversations, unmanaged personal devices |
Read down the blind spot column and a pattern appears. Endpoint detection sees the process but not that it is AI. Device management sees the software but not what it is allowed to reach. The gateway sees the traffic that reaches it and nothing about the traffic that does not. The gap is not an absence of tooling. It is that no existing layer was built to answer the question governance now asks, which is what AI is running here and under what rules.
Why this is a compliance problem, not just a security one
Discovery is often filed under security. The obligations that make it urgent are increasingly written into governance frameworks and law.
ISO/IEC 42001 is the first certifiable AI management system standard, published in December 2023. Certification rests on being able to demonstrate that your management system covers the AI your organisation actually uses. An auditor who finds AI in daily use that sits outside the scope you declared has found a live finding, not a footnote. See our guide to ISO 42001 for how the management system fits together.
The NIST AI Risk Management Framework organises work around four functions: Govern, Map, Measure and Manage. Map comes first for a reason. You establish context and identify the AI in play before you attempt to measure or manage anything. A Map exercise that only reaches sanctioned systems produces a risk picture that is accurate about the wrong population. Our walkthrough of the framework covers the functions in order.
The EU AI Act has been in force since 1 August 2024, with most obligations for high risk systems and the enforcement regime arriving on 2 August 2026, supervised by the European AI Office alongside national market surveillance authorities. Deployer obligations attach to systems you put into use. Determining whether any obligation applies requires knowing what is in use, which returns to the same inventory problem. The regulation tracker holds the current dates and status.
The common thread. Every one of these regimes assumes an inventory. None of them accepts "we could not see it" as a reason a system fell outside scope. Discovery is not preparation for governance work. Under these frameworks it is the first piece of governance work, and everything downstream inherits its coverage.
How to evaluate an endpoint approach
A small category of tools now addresses this layer directly. They are worth assessing against criteria you set before any demonstration, because the demonstrations are uniformly impressive and uniformly show the tool finding things.
What this looks like in practice: Endlayer
Disclosure. Endlayer is built by Holistic AI, the company behind this publication. We have held it to the same criteria set out above, including where it stops.
To see whether the read only, configuration first design holds up, we built the agent from source and ran a single scan on one MacBook. Remote delivery is off until an organisation enables it, so nothing left the machine: the snapshot was written to standard output and read locally. The numbers below are from that scan, and they are one developer laptop rather than a fleet.
It reads state rather than traffic, and it is precise about the boundary. The agent runs as the signed in user and never elevates. Its default interval is 900 seconds, which is the 15 minutes the documentation describes, and the whole scan took 2.3 seconds warm. This build reported 25 signals and, for each one, the documentation separates what is scanned on the machine from what is reported off it. Credential values are discarded before evidence is built: the scan recorded that 18 declarations existed, which providers they belonged to and that two were exposed, without carrying a single value. Tool server declarations were read the same way, with no connection made to any of them.
It captures permission state, not just presence. This is where the scan earned its keep. Counting installed tools told us little. What mattered was that the default agent mode on this machine was unattended, that the approval prompt was set to be skipped, and that 783 standing pre-approvals and 21 hooks had accumulated across 37 project level overrides. Of 184 recorded sessions, 181 had run unattended and one had run with permissions bypassed entirely. None of that is visible to a gateway, an inventory or an endpoint detection tool, because nothing about it is a network request or a suspicious process. It is configuration that nobody had revisited.
Its limits are stated rather than implied. Device collection is available today. Cloud collection from services such as Azure AI Foundry and Amazon Bedrock, and the application SDKs, are documented as planned, and the documentation says so plainly instead of blurring the line. The unmanaged device problem described below applies in full: the agent reaches the machines it is installed on and no others.
Two measurements did not match the documentation, both in small ways worth stating. The published figures are 21 signals and a scan of roughly 320 milliseconds; the build we compiled reported 25 signals and took 2.3 seconds warm, and our unstripped binary was 4.1 MB against a published 3.0 MB. Version drift and a local build rather than the shipped one explain all three, and none of them changes the argument. We note them because a review that reproduces every vendor number exactly has usually not run anything.
Other approaches solve parts of the same problem, and some organisations will close most of the gap by extending tooling they already own: teaching a device management inventory to classify AI software, adding detection rules for credentials in configuration, or requiring managed keys so that personal ones become visible by their absence. The right answer depends on what you already run and on how much of your estate is genuinely managed. Compare any option, including this one, against the criteria above rather than against a demonstration.
The trade-offs nobody puts on the slide
Endpoint discovery closes a real gap. It also brings costs that rarely appear in a pitch.
It is another agent. Endpoint teams have spent a decade consolidating agents. Adding one needs a defensible reason, and performance and stability questions are legitimate rather than obstructive.
Employee monitoring is a regulated subject. In much of Europe, software that observes what employees run engages works council consultation and data protection assessment, whatever its technical design. Reading configuration rather than content is a much easier case to argue, but it is still a case you have to make, and making it early is cheaper than making it late.
Detection lags the ecosystem. New agents appear constantly. Any recognition list trails reality, so coverage is a moving number rather than a state you reach.
Periodic collection has gaps. A tool that samples at an interval sees the estate at intervals. Something installed, used and removed between collections leaves little trace. For inventory this is acceptable. For forensic reconstruction it is not, and the two should not be confused.
Unmanaged devices stay invisible. This is the significant one. Endpoint tooling reaches endpoints you manage. Personal laptops and phones, contractor machines and anything outside enrolment remain outside the picture, which means endpoint discovery reduces the blind spot rather than removing it.
A practical first ninety days
The sequence matters more than the tooling, and the first two steps cost nothing.
Treat the output as an inventory you maintain rather than a report you file. The value is in the second and third collection, when you can see what changed, not in the first, when everything is new and the number is alarming.
Frequently asked questions
Where this leaves you
The uncomfortable part of this shift is that the control that felt complete became partial without anything visibly breaking. Gateway logs still populate, dashboards still show sanctioned traffic, and the number rises quarter on quarter. What changed is the denominator, and nothing in the existing reporting reveals that.
Endpoint discovery is not a complete answer. It does not reach unmanaged devices, its recognition lags the tools it looks for, and it introduces a privacy conversation you have to hold properly. It does close the specific gap that network controls cannot, and it does so with evidence that already exists on machines you already manage. For an organisation heading towards ISO/IEC 42001 certification or preparing for EU AI Act obligations, that is the difference between an inventory you can defend and one you hope nobody examines closely.
Where a governance platform fits. Discovery produces a list. Turning that list into an inventory with owners, risk classifications, framework mappings and audit ready evidence is the work that follows, and it is what an AI governance platform such as Holistic AI is built to carry. The endpoint layer and the governance layer answer different questions, and neither substitutes for the other.
