Entering the Harness Era of Benchmarking: 15 Academic Institutions Jointly Release HarnessEval

On August 13, DeepSeek open-sourced the Agent Harness dsh, pushing "Harness"—a concept originally rooted more in engineering contexts—to the center of discussion across the AI industry.

Today, an Agent's ultimate capability can no longer be simply equated with the capability of the underlying model. Beyond the model, there are context management, tool invocation, memory, task decomposition, execution environment, permission management, and result verification. What truly determines whether an Agent can reliably complete complex tasks is whether these components can be organized into a stable, executable workflow system.

Clearly, understanding Harness merely as a product form of Coding Agent seriously underestimates its significance. Almost all complex intelligent processes require a system layer responsible for planning, coordination, and verification. This is true for Agents, and equally true for Evaluation.

Recently, MirroS, together with Tsinghua University, Peking University, Berkeley, MIT, xbench, NVIDIA, and other research institutions, jointly released HarnessEval, further bringing the Harness concept into evaluation systems. It attempts to answer a more fundamental question than "how to make a model complete a task":

If the AI being evaluated has already changed from a model into a complex system, what kind of Evaluation is sufficiently effective?