The AI Investigation Agent: Automating the Analyst's First Hour

The First Hour Is the Expensive Part

An alert fires. Something is wrong with a server, or a login looks suspicious, or a process is behaving oddly. What happens next, in most organisations, is that a human being opens six or seven browser tabs.

They check the cloud console for recent changes. They SSH into the box and run top and ps. They pull the last day of auth logs. They cross-reference an IP against a threat feed. They squint at a DNS query history. They read a firewall rule. Forty-five minutes later, they have assembled — in their head, or in a hurried notepad file — a picture of what happened. Most of that time was not analysis. It was fetching, formatting, and pivoting between tools that don't talk to each other.

That first hour is where an AI investigation agent earns its place. It is the module that does the fetching and the pivoting automatically, then hands a human a structured, evidence-backed summary with a proposed root cause. It doesn't replace the analyst's judgement. It replaces the tab-switching. This piece is part of the broader AI-native security platform architecture, and sits directly downstream of the detection engine that raises the alert in the first place.

What the Agent Actually Does

The investigation agent is best understood as a small, disciplined workflow rather than a magic box. When a detection fires — or when a human simply asks a question — the agent works through a sequence of evidence-gathering steps, each one a call into the data layer:

• Collect the alert context — what triggered this, on which asset, at what time.
• Collect the process list — what was running on the affected host around that time.
• Collect user and session history — who logged in, from where, and when.
• Collect cloud events — API calls, instance changes, IAM modifications from the provider's audit trail.
• Collect network and firewall logs — inbound and outbound connections, blocked and allowed.
• Collect DNS queries — what the host was trying to resolve, which is often where command-and-control betrays itself.
• Collect threat and malware intelligence — is this IP, hash, or domain known-bad?
• Summarise and propose a root cause — with a confidence rating and the evidence that supports it.

The important thing is that none of these are guesses. Each step is a real query against real data — the same queries an analyst would run by hand, executed in seconds and assembled into one coherent narrative rather than seven disconnected browser tabs.

How the Agent Gathers Evidence

Under the hood, the agent doesn't have free rein over your systems. It has a defined set of tools — strictly typed functions that fetch specific kinds of data. Something like get_process_list(host, time_window), get_cloud_events(account, time_window), get_dns_history(host, time_window), lookup_threat_intel(indicator). The model decides which tools to call and in what order, but it can only call the tools it has been given, and each tool returns only the data it is scoped to return.

This is the same architectural discipline that matters for any AI agent that touches business systems: the model's power comes from reasoning over data, not from open-ended access. The agent is smart about sequencing — if the process list turns up an unfamiliar binary, it will pivot to DNS and network logs for that process rather than blindly collecting everything. But it is fetching through a narrow, auditable interface, and every call it makes is logged.

The reasoning loop is straightforward: look at what's been gathered so far, decide what evidence would be most useful next, call the relevant tool, incorporate the result, and repeat until the picture is complete enough to summarise. A well-built agent knows when to stop — it isn't trying to boil the ocean, it's trying to answer one question.

Learning From History: RAG Over Past Incidents

Raw evidence gathering is only half the value. The other half is context: has anything like this happened before?

This is where retrieval-augmented generation (RAG) over your incident history comes in. Every past investigation — the evidence, the conclusion, the remediation — is embedded and stored. When a new investigation runs, the agent retrieves the most similar historical incidents and folds them into its reasoning. So instead of analysing a spike in isolation, the agent can note: "This pattern of DNS queries to a mining pool matches an incident from three weeks ago on a different host — the root cause then was an exposed Docker API." That's institutional memory that would otherwise live only in a departed analyst's head.

RAG also grounds the agent. Rather than free-associating from its training data, it reasons from your environment's actual history, which sharply reduces the confident-but-wrong failure mode that plagues naive LLM deployments.

A Worked Example: "Why Did the Server CPU Reach 100%?"

Here is the kind of investigation the agent handles end to end. A user — an ops engineer, not a security specialist — types a plain question into the security copilot: "Why did server web-prod-03 hit 100% CPU last night?"

The agent works through it:

The agent's summary, in plain language:

Root cause (confidence: high): web-prod-03 is running an unauthorised crypto-miner. The Docker API on this host was exposed to the internet, which allowed an attacker to launch a container running the miner kdevtmpfsi. A cron job re-installs it every 10 minutes to survive restarts, and it is connecting outbound to a mining pool. This is why CPU is pinned at 100%.

Recommended next steps (for human review): isolate the host from the network, remove the malicious cron entry and container, and — critically — restrict the exposed Docker API, which is the actual entry point. Rotate any credentials that were reachable from this host.

What took the agent under a minute would have taken an analyst most of an hour, and the analyst might not have connected the cron persistence to the exposed Docker API on the first pass. Notice, too, what the agent did not do: it did not kill the process, remove the cron job, or touch the firewall. It investigated and recommended. It did not act.

The Guardrails: Read-Only by Default

That last point is the whole game, and it's worth being emphatic about it.

The investigation agent is read-only. Every tool it holds fetches data. None of them change anything. It cannot stop a process, quarantine a host, revoke a credential, or modify a firewall rule — because it has not been given tools that do those things. This is a deliberate architectural boundary, not a policy you hope the model respects.

Its access is scoped. The agent can read process lists and logs; it cannot read the contents of your database or your customers' files. Each tool is bounded to exactly the telemetry an investigation needs, following the same principle of least privilege that governs securing any AI agent.

Any action requires a human. The output is a recommendation. A person reads the evidence, agrees or disagrees with the proposed root cause, and decides what to do. The confidence rating exists precisely to support that judgement — a "high confidence" root cause backed by seven corroborating pieces of evidence gets treated differently from a "low confidence" hypothesis based on two.

The reason you keep investigation and response separate is not squeamishness — it's blast radius. An investigation agent that got manipulated (through, say, a poisoned log entry crafted to mislead it) can, at worst, produce a wrong summary that a human then sanity-checks. An agent that could both investigate and execute destructive actions could be talked into quarantining a critical production host or revoking a valid admin credential. You never let the thing that reads also be the thing that destroys. Automated response — the playbooks that actually contain a threat — belongs in a separate module with its own, much stricter, approval gates. That's a different conversation, and a different post.

What Good Looks Like

A well-built investigation agent has these properties:

• Every conclusion is backed by cited evidence. The summary links to the actual log lines, process entries, and events it reasoned from. You can check its work.
• It rates its own confidence honestly. It distinguishes a well-corroborated root cause from an educated guess, and says which is which.
• It is read-only, and provably so. Not "instructed not to act" — incapable of acting, because it holds no tools that mutate state.
• Its access is scoped to telemetry. It reads security-relevant logs and metadata, not arbitrary business data.
• Every tool call is logged. You have a full audit trail of exactly what the agent looked at and when.
• It learns from your history. RAG over past incidents means it gets more useful as your environment accumulates resolved cases.

Questions to Ask Before You Build One

"What tools does the agent have, and can any of them change state?" The answer should be a specific list, and every tool should be read-only. If anything on the list can modify, delete, or execute, that's a design you need to challenge.

"How does it show its work?" A root cause with no cited evidence is just an assertion. You should be able to trace every conclusion back to the raw data that supports it.

"How is confidence calculated, and what happens at low confidence?" A good agent hands ambiguous cases to a human clearly labelled as uncertain, rather than presenting a shaky guess with false authority.

"Where does investigation end and response begin?" There should be a hard, architectural line between the agent that recommends and the system that acts. If that line is blurry, the blast radius of a mistake grows dramatically.

What It Costs and How Long It Takes

An investigation agent is not a standalone product — it sits on top of a working data layer. If your logs are already being collected and normalised, building a genuinely useful first version of the agent — evidence-gathering tools for the most common sources, a summarisation loop, and confidence rating — is typically a six-to-ten-week engagement for a small team. Adding RAG over historical incidents comes shortly after, once you have a corpus of resolved cases worth retrieving from.

The honest caveat: the agent is only as good as the data beneath it. If your process lists are patchy or your cloud audit trail has gaps, the agent will faithfully investigate incomplete evidence and reach shakier conclusions. Most of the real engineering effort in this module is not the AI — it's the plumbing that makes clean, queryable telemetry available for the tools to fetch. Budget accordingly, and be sceptical of any timeline that assumes the data layer is already perfect.

Related guides

• Building an AI-native security operations platform
• Building the detection engine that raises the alerts
• The AI security copilot: natural-language queries over your data
• Multi-agent architecture for security operations
• AI agent security: what business owners need to know
• Our AI agent development services

We Build Investigation Agents That Know Their Limits

The investigation agent is one of the most satisfying modules to build well, because the value is so immediate and the guardrails are so clear. We'd start by scoping the evidence-gathering tools around your actual sources, wiring in a read-only data layer, and proving out the worked-example flow — an alert in, a cited, confidence-rated root cause out — before layering in RAG over your incident history.

Crucially, we build it read-only by design and keep investigation cleanly separated from response, so that automating the analyst's first hour never means handing an unsupervised agent the keys to your production estate.

Talk to us about your platform — no commitment, just a conversation.

Frequently Asked Questions

What is an AI investigation agent in security operations?

It's an AI system that automates the evidence-gathering an analyst does at the start of an investigation. When an alert fires, the agent calls a set of read-only tools to collect the relevant process lists, logs, cloud events, DNS history, and threat intelligence, then reasons over that evidence to produce a summarised, confidence-rated root cause. It replaces the tedious pivoting across consoles that eats the first hour of most investigations — but it recommends rather than acts, leaving the decision and any remediation to a human.

How is an investigation agent different from an automated response system?

Investigation agents read and recommend; response systems act. The investigation agent gathers evidence and proposes a root cause, but it holds no tools that can change anything — it cannot kill processes, quarantine hosts, or revoke credentials. Automated response (a SOAR-style module) runs containment playbooks that do change state, and it lives behind much stricter approval gates. Keeping the two separate is a deliberate safety boundary: the thing that investigates should never also be the thing that can destroy, because that dramatically increases the blast radius of any mistake or manipulation.

Can I trust the root cause the agent proposes?

Trust it the way you'd trust a junior analyst's write-up — as a well-organised starting point, not a final verdict. A well-built agent cites the specific evidence behind every conclusion and rates its own confidence, so you can check its work rather than take it on faith. High-confidence conclusions backed by several corroborating pieces of evidence are usually reliable; low-confidence hypotheses are flagged as such and handed to a human. The agent is a force multiplier for analyst judgement, not a replacement for it.

Why does the investigation agent have to be read-only?

Because an agent that can both investigate and take destructive action is far more dangerous if it's ever wrong or manipulated. A read-only agent that's fed a misleading log entry can, at worst, produce a wrong summary that a human sanity-checks. An agent with the power to act could be talked into quarantining a critical production server or revoking a valid credential. By giving the investigation agent only tools that fetch data — and none that change state — the worst-case outcome is a bad recommendation, not a bad action. This is enforced architecturally, not by trusting the model to behave.

What data does the investigation agent need to work well?

It needs clean, queryable telemetry: host process lists, authentication and session logs, cloud provider audit trails, network and firewall logs, DNS query history, and a threat intelligence feed. The agent is only as good as this underlying data — gaps or patchy collection lead to shakier conclusions. In practice, most of the engineering effort in building an investigation agent goes into the data layer that makes this telemetry reliably available, not into the AI reasoning on top. Adding RAG over your own resolved-incident history further improves results by grounding the agent in your environment's actual patterns.

How long does it take to build an investigation agent?

Assuming your logs are already being collected and normalised, a useful first version — read-only evidence-gathering tools for your most common sources, a summarisation and reasoning loop, and confidence rating — is typically a six-to-ten-week engagement for a small, focused team. Adding retrieval over historical incidents follows once you've accumulated a corpus of resolved cases. The timeline stretches considerably if the data layer isn't ready, because the plumbing that supplies clean telemetry is where most of the real work lives.