Building an Adaptive Agentic Cybersecurity System with NVIDIA Nemotron

The experiment automates the handoff that normally kills red-and-blue iteration speed. A red-agent harness executes an attack in an isolated environment modeled on NVIDIA's accelerated computing infrastructure; Falcon sensors capture telemetry; a blue-agent harness reconstructs the sequence, finds gaps, and writes candidate detections; validation backtests them; then an independently seeded attack retests and the red agent looks for another path.

The defensive stack split roles. Nemotron 3 Ultra handled orchestration — reconstructing sequences, planning, calling tools. A customized Nemotron 3 Super, used by CrowdStrike as the base for its NL2LogScale model, acted as a bounded expert for writing and repairing detections. Fine-tuning used 9,349 detection-generation and multistep repair examples spanning 59 programmatically generated error types, with reinforcement learning through NeMo Gym and NeMo RL where the reward was F1 overlap between the generated query's returned events and a reference query's.

The headline number is the wrong one to fixate on. Ultra with a default harness detected the recorded attack 16.5 percent of the time; adding the tuned harness, the specialized Super, domain context, tools, and validation lifted that to 41.9 percent, a 2.5x improvement. NVIDIA is careful to say the gain reflects the whole pipeline, not an isolated model swap. That caveat is the finding: harness engineering did as much work as the models.

Live fire is where it gets interesting. Eleven backtest-passing open detections went up against eight unseen attacks alongside 35 from the frontier system. Five of eleven open detections fired, 45 percent, versus ten of 35 frontier detections, 29 percent. After the quiet-on-benign and independent-review gates, three open detections qualified as gold and zero frontier detections did — and those three still covered all eight attacks.

Signal Desk's read: the frontier system was better at producing things that looked like detections and worse at producing detections that survive contact with a variant. That is a familiar failure mode. Fluent output passes a backtest; grounded output generalizes. The six mechanisms in the defensive harness — schema enumeration, telemetry grounding, artifact linting that rejects hardcoded IPs and hostnames, replay, and a fresh-context judge — are doing the work a senior detection engineer does in review, and they are the reason the open pipeline's survivors were behavioral rather than environment-specific.

The honest limits are stated in the post and worth repeating. One scenario family. Small detection sets. Limited benign traffic, so the noise test says nothing about production false positives. Three of eight live-fire runs hit harness failures and were retained anyway. This is a directional case study, not a benchmark, and anyone quoting 45 percent versus 29 percent without those numbers is selling something.

CrowdStrike says its Blue Solano defensive model beat the leading proprietary frontier model it tested at 99 percent lower cost, in internal evaluations. That cost line is the commercial argument underneath the whole exercise: if a post-trained open model can be the bounded expert, detection engineering stops being priced per frontier token.