AI companies rely on benchmark tests to show that their models perform better than rivals. The methods behind those tests are still developing, according to Hugging Face’s chief policy officer.
Irene Solaiman spoke about the issue during a discussion at TechCrunch Disrupt last week. She said testing needs to become a more consistent science if the industry plans to depend on it.
Can AI Test Results Be Trusted?
Asked whether current evaluations can be trusted, Solaiman said, “Trust is such a weighted term.”
She said one major part of trust is whether the results of an evaluation can be reproduced. Another is whether testers have enough access to the model itself.
“This is where open weights come in in a big way,” she said. Open weights refers to AI models whose underlying parameters are made public.
Reproducibility has long been one of the harder parts of AI benchmarking. It remains a debated topic in the field.
Solaiman also pointed to the challenge of comparing different tests. “They are an evolving science,” she said, adding that “we don’t know how to meaningfully compare them to each other.”
She said tests “start to get ineffective relatively quickly.” She added that the field does not yet have the “thriving ecosystem” it needs.
The Push for Embedded Evaluators
Embedded evaluators are third-party observers placed inside AI companies to review their work. The idea has gained attention over the last several months.
AI labs have backed the approach following a series of safety incidents that raised concerns about possible harm from their technology. Hugging Face itself was the target of one of those incidents.
Labs have promoted third-party evaluators as an alternative to active government regulation. Questions remain over how neutral and effective these evaluators will be.
Solaiman said there is a “hunger” for an embedded evaluator ecosystem right now. She said the qualifications for who makes an effective evaluator, and for which tasks, need much more standardization.
Anthropic faced backlash when it hired Accenture as its first embedded evaluator. Some critics online mocked the idea that the consulting firm was qualified for the job.
Testing is also expanding beyond the model alone. Evaluations now look at the whole system around it, including the harness and the agent.
The harness is the tool a person uses to access a model. Solaiman said a weaker model with a better harness could outperform a leading frontier model.
“Agentic evals are really hard and really expensive,” she said. She noted that there are far more evaluations for models than for harnesses.
Solaiman said better evaluations are now being built for complete AI agents. She said that is where many users are getting the most value.
