Support New AI Frameworks Faster Using NVIDIA Workload Map

Anyone running AI training or inference on Kubernetes knows the pain: every framework, whether JobSet, Ray, KServe or LeaderWorkerSet, describes its pods, replicas and status differently. Schedulers and dashboards need custom adapter code for each one, so supporting a new framework turns into a roadmap item.

Workload Map, formerly called Karta, moves that knowledge into declarative definitions that can be written and reviewed like any other manifest. NVIDIA ships a catalog of prebuilt definitions, released it under Apache 2.0, and says it already runs in production inside Run:ai. That lowers the cost of letting teams bring the frameworks they actually want.

We like this one. It's practical infrastructure work that removes duplicated effort. The JobSet example makes the point: without a definition, a platform sees three separate jobs asking for four GPUs each. With one, it sees a single workload with a leader and two workers asking for twelve. That difference matters for scheduling, cost tracking and simply understanding what's running.

The design choice we find most sensible is treating workload structure as data rather than compiled code. A team can add a custom workload type by writing YAML, and KAI Scheduler uses these definitions as a fallback while native plugins still take precedence. That's a careful way to add flexibility without breaking what already works.

There's a strategic layer worth naming. Workload Map is part of NVIDIA's DSX AI Factory platform, and its first consumers are NVIDIA's own Run:ai and KAI Scheduler. Open licensing helps, and Elotl's Nova adopting it for multi-cluster scheduling is a good sign of use beyond NVIDIA. But whoever defines the common description of AI workloads gains influence over how clusters are run, and today that's NVIDIA.

The practical risk is drift. Definitions rely on paths and jq expressions into each framework's spec. When upstream projects change their schemas, someone has to keep the catalog current, or tools will quietly misread what's running. We'll be watching whether maintenance is shared with the community or stays an NVIDIA job.

The project ships as a Kubernetes custom resource plus a Go library, with a command-line tool for inspecting live workloads. Its catalog covers JobSet, PyTorchJob, RayCluster, RayJob, RayService, KServe, Knative, MPIJob, NVIDIA NIM, LeaderWorkerSet, Milvus, Grove PodCliqueSet and NVIDIA Dynamo.