Embedded AI evaluators won’t be an easy solution, Hugging Face’s policy chief says

Benchmarks have become the scoreboard of the AI race. Labs cite them in launches, buyers lean on them in procurement, and policymakers increasingly treat third-party testing as a substitute for regulation. At TechCrunch Disrupt last week, Irene Solaiman, Hugging Face's chief policy officer, argued that the science underneath all of that is still young. Results are hard to reproduce without deep model access, tests are hard to compare with one another, and many go stale fast.

That matters right now because the industry's favorite answer to a run of safety incidents is the embedded evaluator: a supposedly neutral outside party placed inside a lab. Solaiman's point is that there is real hunger for that ecosystem, but no shared standard for who qualifies to evaluate what. If the referees are not credentialed, the whistle does not mean much.

We think Solaiman is right, and we think this is one of the more important unglamorous fights in AI. Third-party evaluation is a genuinely good idea. It is how we trust bridges, drugs, and financial statements. But auditing works because the auditor's methods are standardized and their independence is checkable. AI evaluation has neither yet.

The reproducibility point is the sharpest one. If an outside tester cannot rerun a result because they lack access to the model itself, then what the public gets is a press release with a number in it. Solaiman tied that directly to open weights, and while that is an argument Hugging Face naturally favors, it is also simply true: you can check what you can inspect. Closed labs will need to offer evaluators far more access than they do today if these arrangements are going to carry weight.

The second point deserves more attention than it gets. Evaluation is shifting from the model alone to the whole system around it: the harness, the tools, the agent loop. Solaiman noted that a weaker model in a better harness can beat a frontier model. That cuts both ways for safety. A model that tests clean in isolation can behave very differently once it is wired into tools and given permission to act. And agentic evaluations, by her account, are hard and expensive, which means fewer of them get done.

Here is the downside if this scales badly. Labs adopt embedded evaluators as a credibility badge, regulators accept that badge in lieu of rules, and the evaluations themselves stay inconsistent, unreproducible, and narrow. That is not oversight. That is outsourcing reassurance. The backlash over Anthropic choosing Accenture as its first evaluator is an early signal that the public is already skeptical of who gets picked and why.

I'm watching for whether anyone moves to define evaluator qualifications in a way that is public and testable, and whether labs commit to the level of access that would make outside results reproducible. Without both, embedded evaluation risks becoming theater, and the useful version of this idea gets discredited along with it.

Embedded evaluators, meaning outside observers placed inside AI organizations, have gained momentum over recent months as labs face safety incidents and push third-party checks rather than direct government regulation. Anthropic's selection of Accenture as its first evaluator drew criticism over whether a consulting firm is qualified for the work.