The Layer That Decides What Matters
Every other part of a security platform exists to feed this one. Collection gathers the events, processing normalises them, threat intelligence enriches them with context — and then the detection engine has to answer the only question that counts: out of the millions of events that arrived today, which handful should a human actually look at?
Get this wrong and nothing else matters. A platform that surfaces a thousand alerts a day is functionally identical to a platform that surfaces none — because a team that can't triage a thousand alerts will triage zero. The detection engine is where a security product either earns trust or quietly loses it.
This is the heart of the system. It's also the part where "AI security" is most often oversold and least often built well. Below is what a detection engine actually does, how the different detection strategies fit together, and why the hardest problem isn't catching attacks — it's not crying wolf.
What a Detection Actually Is
At its simplest, a detection is a statement of the form: when this pattern appears in the event stream, raise an alert. The pattern might be a single event ("a user was added to the domain admins group") or — far more often — a correlation across many events over time ("more than 20 failed SSH logins from one source within five minutes").
That word correlation is the whole game. Individual events are almost always innocent. One failed login is someone fat-fingering a password. One large file upload is someone sending a video. One PowerShell command is an admin doing their job. Attacks reveal themselves in sequences and volumes — the failed logins that suddenly succeed, the upload that's a thousand times larger than usual, the PowerShell that downloads and then executes.
So the correlation engine is the core primitive. It holds a rolling window of normalised events in memory (or in a fast store like ClickHouse) and continuously evaluates conditions against that window: counts within a time span, sequences in a particular order, the same actor appearing across different event types. When a condition matches, it emits a detection — not yet an alert a human sees, but a candidate.
Three Ways to Detect, and Why You Need All Three
There is no single detection technique that catches everything. A serious engine layers three, because each covers the blind spots of the others.
Static rules are explicit, human-written conditions. "More than 20 failed logins in five minutes." "A process named powershell.exe spawned by a Word document." "A login from a country we've never seen for this user." Rules are fast, cheap, and — crucially — explainable: when one fires, you can say exactly why. Their weakness is that they only catch what someone thought to write down. A novel attack that doesn't match any rule sails straight through.
Behavioural analytics fix part of that gap by learning what "normal" looks like per user and per asset, then flagging deviations. The engine builds a baseline: this employee usually logs in from London on weekdays, touches three internal systems, and uploads around 100MB a week. When the same account suddenly uploads 120GB overnight from a new location, no static rule needs to exist — the deviation from the baseline is itself the signal. This is often called UEBA (user and entity behaviour analytics), and it's where a lot of the genuine value sits, because it catches misuse of legitimate credentials, which static rules struggle with.
Anomaly detection is the statistical and machine-learning layer that catches what neither rules nor per-user baselines can express. It looks across the whole population for outliers: a server whose outbound traffic pattern doesn't match any of its peers, a login time that's a statistical anomaly for the department, a rare parent-child process relationship that almost never occurs in the estate. This is the fuzziest layer — the most prone to noise — and it works best as a ranking signal rather than a standalone alarm.
The mistake is treating these as competitors. They aren't. Static rules give you precision and explainability, behavioural analytics catch credential misuse, and anomaly detection surfaces the genuinely novel. A mature engine runs all three and lets them reinforce each other — a static rule that fires and corresponds to a behavioural anomaly is a much stronger signal than either alone.
Example Detection Rules
Concrete rules make the idea tangible. Here are four that most engines ship early, spanning the categories above, with the MITRE ATT&CK technique each maps to (more on that mapping below).
| Detection | Condition | Type | MITRE ATT&CK | Typical severity |
|---|---|---|---|---|
| SSH brute force | More than 20 failed logins from one source IP within 5 minutes | Static rule | T1110 Brute Force | Medium |
| Impossible travel | Successful login from India, then a successful login from Germany 10 minutes later — physically impossible | Behavioural | T1078 Valid Accounts | High |
| Data exfiltration | User who normally uploads ~100MB uploads 120GB in a day | Behavioural (UEBA) | T1567 Exfiltration Over Web Service | High |
| Suspicious process chain | powershell.exe downloads a payload, then executes a Base64-encoded command | Static + anomaly | T1059 Command & Scripting Interpreter | Critical |
Notice that the impossible-travel rule isn't magic — it's geometry. Two successful logins, two geolocations from enrichment, and the distance divided by the time gap exceeds any plausible travel speed. That's it. Most powerful detections are this mundane underneath, which is exactly why explainability matters: a security lead can understand and trust "these two logins are 6,000km apart, ten minutes apart" far more readily than "the model scored this 0.87."
Risk Scoring: Turning Detections Into a Queue
If every detection produced an equal-priority alert, you'd be back to the thousand-alerts-a-day problem. So the next job is ranking. Every detection gets a risk score, and the engine surfaces the top of the queue.
A workable risk score combines several factors: the inherent severity of the detection type, the value of the asset involved (a detection on a payroll server outranks the same detection on a test box), the sensitivity of the identity (an admin account outranks a contractor), corroboration from threat intelligence (is that source IP a known-bad indicator?), and — importantly — whether multiple detections cluster around the same entity in a short window. Ten weak signals on one host in an hour is a stronger story than one medium signal in isolation.
Scoring is what lets a small team operate the platform. Instead of "here are 400 alerts, good luck," the interface says "here are the six things most likely to be real, ranked, with the evidence attached." That framing is the difference between a tool a generalist IT lead can run and one that needs a trained analyst.
The Real Enemy: False Positives and Alert Fatigue
Here is the honest centre of the whole subject. A detection engine that cries wolf is worse than none at all. A platform with no detections gives a team an accurate sense of its own blindness. A platform that fires forty false alarms a day trains the team to ignore alerts entirely — and the one real alert arrives in a stream the team has already learned to dismiss.
Alert fatigue is not a minor tuning issue. It is the failure mode of security operations, and it's why so many expensive deployments end up as dashboards nobody reads. Every over-eager rule, every anomaly detector left at a hair-trigger threshold, every alert without enough context to act on, contributes to the erosion of trust that makes the whole system worthless.
Fighting it is continuous work, not a one-off. Rules need tuning against the specific environment — the "20 failures in 5 minutes" threshold that's right for a small office is wrong for a service that legitimately retries. Behavioural baselines need a learning period and a way to absorb legitimate change (a genuine office relocation shouldn't fire impossible-travel forever). And there needs to be a feedback loop: when an analyst marks an alert as a false positive, that judgement should feed back into scoring, so the engine gets quieter and sharper over time rather than louder.
Mapping to MITRE ATT&CK, and Where AI Triage Sits
Two things separate a serious engine from a pile of ad-hoc rules.
The first is mapping every detection to MITRE ATT&CK — the industry-standard catalogue of adversary techniques (the T-numbers in the table above). This isn't box-ticking. It gives you a coverage map: which techniques can we actually detect, and where are the gaps? It gives every alert a shared vocabulary, so "T1078 Valid Accounts" means the same thing to your engine, your analyst, and the threat report they read yesterday. And it lets detections chain into a narrative — a brute force (T1110) followed by valid-account use (T1078) followed by exfiltration (T1567) reads as a coherent attack, not three unrelated blips.
The second is AI triage sitting on top of the raw detections, before a human ever sees them. This is where an AI-native platform genuinely differs from a bolted-on chatbot. When a detection fires, an investigation agent gathers the surrounding evidence — recent activity for that user, the process tree, related events on the same host, the reputation of the IP — and produces a plain-language assessment: this looks like a real brute-force attempt that succeeded, here's the evidence, here's the likely next step. Or, just as valuable: this is almost certainly benign — a monitoring script retrying — here's why, deprioritised.
That triage layer is the single biggest lever on alert fatigue. It doesn't replace the detection engine; it filters and enriches its output so that the alerts reaching a human are already contextualised and ranked. The detection engine decides what is suspicious. The AI decides what's worth your attention right now — the same discipline that matters when securing any AI agent that acts on real systems.
What Good Looks Like
A detection engine you'd trust in production has these properties:
Layered detection. Static rules, behavioural baselines, and anomaly detection all running, reinforcing each other rather than competing.
Explainable alerts. Every alert says why it fired, in terms a non-specialist can follow — not just a score.
Risk-ranked output. The interface leads with the few things most likely to be real, not an undifferentiated list.
MITRE mapping. Detections carry ATT&CK technique IDs, giving a coverage map and a shared vocabulary.
A false-positive feedback loop. Analyst judgements feed back into scoring so the engine sharpens over time.
AI triage in front. Raw detections are enriched and assessed before a human sees them, so the queue is short and contextualised.
Questions to Ask Before You Build One
"How do you tune out false positives, and how does that improve over time?" If there's no feedback loop from analyst decisions back into scoring, the engine will get louder, not quieter.
"Which detections are static rules versus behavioural versus anomaly-based?" A serious answer names all three and explains what each covers. "It's all AI" is a red flag, not a feature.
"Can every alert explain why it fired?" If the only answer is a model score, your team will never trust it — and untrusted alerts get ignored.
"How are detections mapped to a framework like MITRE ATT&CK?" This is how you know your coverage and your gaps, rather than guessing.
What It Costs and How Long It Takes
Honest framing: a basic detection layer — a set of well-chosen static rules over a working processing pipeline, with simple risk scoring — is typically a four-to-six-week piece of work once collection and processing are in place. That alone catches a meaningful share of the obvious attacks and is genuinely useful.
Behavioural analytics add more: you need a learning period to build baselines, and tuning time to get the thresholds right for the specific environment. Realistically that's a further six-to-ten weeks, and it keeps improving with feedback rather than being "done." Anomaly detection and AI triage sit on top of that and are best added once the fundamentals are stable — building them first, before you have clean data and tuned rules, just produces expensive noise.
The most expensive mistake is optimising for detection coverage over detection precision early on. A demo that fires on everything looks impressive and is useless in production. The engineering effort that matters most isn't writing more rules — it's the tuning, scoring, and triage that keep the alert queue short enough that a real team will actually work it.
Related guides
- The full AI-native security platform architecture
- Threat intelligence enrichment: turning indicators into context
- The AI investigation agent: auto-gathering evidence
- The log processing pipeline that feeds detection
- AI agent security: what business owners need to know
- Our AI development services
We Build Detection Engines That Stay Quiet Until They Shouldn't
The hard part of a detection engine isn't catching attacks in a demo — it's building one a small team will still trust in month nine, when the novelty has worn off and the alert queue has to be short, ranked, and explainable every single day. That's a data-engineering and tuning problem as much as an AI one, and it's exactly the kind of system we like to build: real correlation underneath, behavioural baselines in the middle, and AI triage on top that keeps the noise off a human's desk.
If you're scoping a security platform, or trying to fix a detection layer that's drowning your team in false positives, we're happy to walk through your specific case.
Talk to us about your platform — no commitment, just a conversation.
Frequently Asked Questions
What is a security detection engine?
It's the component of a security platform that turns a stream of normalised events into a small number of meaningful alerts. It runs conditions and correlations against the event stream — counts within a time window, sequences of actions, deviations from normal behaviour — and emits a detection whenever a suspicious pattern appears. It sits downstream of collection and processing and upstream of investigation and response, and it's the layer that decides which of the day's millions of events a human should actually look at.
What's the difference between rules, behavioural analytics, and anomaly detection?
Static rules are explicit human-written conditions ("more than 20 failed logins in five minutes") — fast, precise, and explainable, but they only catch what someone thought to write down. Behavioural analytics learn what's normal per user and asset, then flag deviations, which catches misuse of legitimate credentials that no static rule would spot. Anomaly detection is a statistical or machine-learning layer that finds outliers across the whole population — the novel things neither of the others can express. A good engine runs all three, because each covers the others' blind spots.
What is impossible-travel detection?
It's a detection that flags two successful logins for the same account from locations too far apart to have travelled between in the elapsed time — for example, a login from India followed ten minutes later by one from Germany. Underneath it's simple geometry: two geolocations from enrichment, the distance between them, divided by the time gap. If the implied speed exceeds any plausible travel, it's physically impossible, so one of those sessions is almost certainly using stolen credentials.
Why are false positives such a big problem?
Because a detection engine that cries wolf is worse than none at all. A platform with no alerts at least gives a team an accurate sense of its own blind spots. A platform that fires dozens of false alarms a day trains the team to ignore alerts entirely — and then the one real alert arrives in a stream everyone has already learned to dismiss. Alert fatigue is the single biggest failure mode of security operations, which is why tuning, risk scoring, and AI triage matter more than raw detection coverage.
What is MITRE ATT&CK and why map detections to it?
MITRE ATT&CK is an industry-standard catalogue of adversary techniques, each with an identifier (like T1110 for brute force or T1078 for valid-account abuse). Mapping every detection to it gives you three things: a coverage map showing which techniques you can and can't detect, a shared vocabulary so an alert means the same thing to your engine and your analyst, and the ability to chain detections into a coherent attack narrative rather than a list of unrelated blips.
How does AI triage reduce alert noise?
AI triage sits on top of the raw detections, before a human sees them. When a detection fires, an investigation agent automatically gathers the surrounding evidence — the user's recent activity, the process tree, related events on the same host, the reputation of the IP — and produces a plain-language assessment of whether it's likely real or benign, with the reasoning attached. The detection engine decides what is suspicious; the AI decides what's worth attention right now. Done well, it's the single biggest lever for keeping the alert queue short enough that a small team will actually work it.