How NVIDIA OpenShell Reference Architecture Secures and Governs AI Agent Workloads at Scale

NVIDIA laid out a reference architecture for running autonomous agents in production. At its core is OpenShell, an open-source runtime that treats every agent workload as untrusted and decides, from outside the agent, which files, services and devices it can touch. It has four parts: a gateway that manages sandboxes and policy, a supervisor that checks outbound requests and brokers credentials, the sandbox itself, and a policy prover that formally checks a policy stays under an organization's ceiling.

Around it, NVIDIA stacks its own hardware: Vera CPUs to run the agent loops, and BlueField-4 data processing units running NVIDIA Sentry, which watches agent behavior and can quarantine an agent within milliseconds.

This matters because agents that run for hours, download code and spawn other agents create compounding risk. Where the guardrails live is now an infrastructure decision, not a prompt decision.

We think the core design principle is right, and overdue. The post states it plainly: security and governance belong in the infrastructure, not inside the agent. Models can be talked out of their instructions. A supervisor process that holds the credentials and checks every outbound request against policy cannot be sweet-talked the same way. Formal verification of policy limits is a genuinely useful addition, because misconfigured permissions are where real incidents start.

Treating the agent as untrusted from start to finish is the healthy posture, and we would like to see it become the default rather than the premium tier.

Now the part to read carefully. OpenShell is open source, but the reference architecture is a sales map. The performance claims belong to the Vera CPU: up to 2x on agentic workloads versus x86, 2x performance per watt, and CoreWeave's report of more than 3x faster sandbox startup, 1.8x faster agent loops on Terminal Bench, and over 11,000 concurrent agent environments in one rack. Those are vendor and partner numbers against an unnamed x86 baseline. The hardware enforcement layer depends on BlueField-4. Governance that only reaches full strength on one company's silicon is still governance, but buyers should know they are choosing a stack, not just a policy engine.

There is a harder question underneath. Sandboxes and egress rules stop agents from reaching where they should not. They do less about an agent doing the wrong thing with access it legitimately holds, like a bad refund or a destructive database command inside its allowed scope. NVIDIA nods at this, pointing to identity, authorization and auditing systems that can plug in. That integration work is where most enterprises will actually succeed or fail.

Our read: useful plumbing, the right philosophy, and a reminder that agent safety is becoming a hardware market.

OpenShell supports middleware that can inspect or modify an agent's network traffic and interceptors that add governance to control-plane operations. NVIDIA positions the Vera Rubin platform, OpenShell, and Sentry on BlueField-4 as the three pillars of the design.