Contrastive-LM Releases CLM-8B: An Open System One Model That Scores Agent Actions Up to 9× Faster Than Jev

Agent stacks spend a lot of their wall clock not writing prose but picking the next tool, next move, or next candidate. CLM-8B is the first open Contrastive Language Model aimed at that job: it scores candidate actions against state and returns probabilities instead of generating tokens.

MarkTechPost’s write-up puts the Apache-2.0 projection head at about 75 MB on a frozen Qwen3-8B backbone, runnable on a single NVIDIA GPU under Linux with vLLM. If open System One scoring holds up outside the team’s tables, the bottleneck for self-hosted agents shifts from “can we afford a proprietary scorer” to “can we keep the action set cached.”

We’re reading this as infrastructure for useful agents, not another chat model drop.

CLM trains state and action encoders with bidirectional InfoNCE, then scores by embedding dot product and softmax. That design matches how agent loops actually behave: state changes every step while the action menu mostly stays put. The team’s clm-serve path caches vectors the way vLLM caches KV — on one RTX 4090 with three actions, revisited states drop from 1.7 ms to 0.6 ms. The flashy 9× figure versus Jev comes from the T-Rex game, where actions repeat; the model card also cites about 13× with roughly a thousand candidates.

Zero-shot, CLM matches Jev on T-Rex and Super Mario and trails on BFCL v4 tool calling (95.2% vs 99.2%) and WikiRacing (26/30 vs 30/30) while staying faster on every listed task. As a fine-tuned verifier on held-out DeepSWE and Terminal-Bench 2.1 subsets, CLM beats pass@1 and Jev — 81.6% vs 73.7% pass@1 and 71.1% Jev on DeepSWE; 87.6% vs 84.0% and 83.1% on Terminal-Bench — at 4.1× to 5.7× lower latency on an H100. Those are subset claims with lightweight heads, not full leaderboard submissions, and the authors note Jev scoring below pass@1 means picking with Jev can be worse than taking one sample.

Useful AI loves an open scorer you can host next to your tools. The downside is familiar: latency wins on toy games and held-out slices can evaporate when action vocabularies drift or when hard negatives from Gemini 2.5 Flash-Lite stop matching production mess. Training staged ~60M Nemotron DQA pairs, ~30M synthetic hard negatives, then ~1M agent trajectories — and mid-training hard negatives after pre-training beat starting with them (69.2% vs a 62.4% peak that overfit).

I’m watching whether TypeSafe-compatible API replay becomes the interchange format for System One, and whether independent coding-agent teams reproduce the verifier SOTA outside the authors’ 38- and 30-task subsets. If they do, open action scoring gets cheap enough to sit in every serious agent loop. If they don’t, CLM stays a strong research baseline with a marketing 9×.

Michal Sutter for MarkTechPost, Sep 23, 2026, covering Contrastive-LM’s CLM-8B release, training recipe, zero-shot latency tables versus TypeSafe AI’s Jev (limited early access Sep 15, 2026), and fine-tuned verifier results on DeepSWE and Terminal-Bench 2.1 held-out subsets.