The Detection-Validation Loop: From "I Ran It" to "We Caught It"

In Part 1 I described Cyllex as "execute a technique, then check whether your defenses saw it." This post is about that second clause — the detection-validation loop — because it's what separates a purple-team tool from an attack tool.

Logged vs. Detected: Two Different Questions

Before anything else, a distinction that matters more than people expect. When you run a technique, there are two things you might be validating:

• Was it logged? — did the telemetry that should describe this action actually reach your SIEM? (Process creation events, sign-in logs, audit records.)
• Was it detected? — did an analytic rule fire and raise an alert?

These are not the same. You can have perfect logging and zero detections, or an alert that fired off a single weak signal. Cyllex treats telemetry as the ground floor — if the events aren't there, no rule can ever catch it — and builds the verdict up from there.

How the Loop Works

Every technique in Cyllex ships with detection metadata: the telemetry it's expected to produce — Windows/Sysmon event IDs, cloud audit events, the log table and fields. After a campaign runs, Cyllex uses that metadata to go ask your SIEM a precise question, scoped to the exact time window of the execution.

1. Execute the technique (agent, agentless, or cloud).
2. Wait for the SIEM to ingest events (seconds for endpoint, minutes for cloud).
3. Correlate — Cyllex queries the SIEM for the expected telemetry in the execution window.
4. Classify — turn the result into a per-technique verdict.

The Four-State Verdict

Each technique lands in one of four states, so the result is honest rather than binary:

• Detected — the expected telemetry (or an alert) was found.
• Partial — some expected signals showed up, but not the strongest ones.
• Not Detected — the action happened but nothing was logged. A real gap.
• Unknown — the technique has no detection metadata to check against, so Cyllex won't pretend to have an answer.

That last state matters: a tool that reports "Not Detected" when it simply didn't know what to look for produces misleading coverage numbers. Cyllex keeps "we couldn't check" separate from "your SOC missed it."

Two Detection Models, One Engine

Not all telemetry behaves the same, so Cyllex classifies with two models depending on the data source:

• Strong-signal (endpoint event IDs): a single high-value signal — process creation, LSASS access, PowerShell script-block logging — is enough to count as detected; weaker-only signals are partial.
• Audit-count (cloud audit logs): each expected API event is an equal-weight signal, so any match — or an external alert — means detected.

Same loop, same report; the classifier just reflects how each kind of telemetry actually works.

From Gaps to Fixes

A gap report is only useful if it tells you what to do next. For undetected techniques, Cyllex drafts Sigma rule suggestions and maps findings to detection-maturity scoring and control frameworks — so the output is a prioritized to-do list for the blue team, not just a red/green grid.

Correlation can run automatically after a campaign, so the loop is hands-off: schedule the run, and the detection report is waiting for you.