Infini-AI-Lab Releases Hermit, an Open-Source Harness Adapter Framework for AI Agents

Most teams trying to make a coding agent better at their own work have two bad options: fine-tune a model they may not control, or fork a harness they don't own. Infini-AI-Lab's Hermit offers a third. It trains an external "harness adapter" that holds instructions, skills, memory, tools, and coordination logic, while the base model and the agent harness stay frozen.

The first release, v0.1.0, is Apache-2.0 and installs from PyPI. Trained adapters can be deployed to Codex, Claude Code, or Pi, or exported as native plugins. If the approach holds up, adapting an agent to a new job looks more like shipping a versioned config than running a training job.

We like the shape of this. The industry has spent two years treating the model as the only thing worth tuning, while the scaffolding around it, the prompts, skills, memory, and tool wiring, does a lot of the real work. Hermit treats that scaffolding as something you can train, check against a template, and roll back. Every adapter version lives in a git tree. That is the kind of boring discipline that makes agents usable inside real companies.

The design argument is also sound. Many harnesses are closed, so you can't edit them anyway. Editing open ones can break interactions their builders tuned carefully. And an adapter that moves between compatible models is a hedge against vendor churn, which matters when model releases arrive every few weeks.

Now the caveats, and they are big ones. Every reported number uses a single acting model, GLM 5.3 Flash. The results are self-reported, mostly from single runs. And the spread is wide: the best adapter method hits 0.795 on one CL-Bench task against a 0.580 baseline, while the prompt-only adapter lands at 0.470, below that baseline. So the method you choose matters a lot, and a bad choice can make an agent worse. The retail-support benchmark makes the same point from another angle: the best method still sits far below an oracle that is simply told the policy.

There is a quieter risk too. An adapter that learns from trajectories and judge feedback is only as good as the judge. Point it at a sloppy scorer and it will learn to please the scorer. Sandboxing for learner programs is there, on Linux with bubblewrap, which is good, but teams should treat a trained adapter like any other code that touches production: reviewed, versioned, and tested.

Our read: this is a useful idea at the right layer, and the open license makes it easy to test. I'm waiting for independent runs on other models before calling it a step change.

Hermit's training loop is RL-style: a trainer runs tasks through the agent with the current adapter, a judge scores the output, and an evolver proposes a new adapter version. Five evolution methods ship today, including GEPA, SkillOpt, and a MetaWorld Model approach, with more listed as coming soon. CMU postdoc Haizhong Zheng announced the release on X on October 27.