PoWerHiveMind

PowerWorld Bench

30 real PowerWorld study requests · a fresh Claude every time · graded against a key written before the run

Claude Opus 5.5 · the same 30 tasks for every setup · whiskers show the 95% interval

RankSetupSolvedSolved, as a barMin / taskTokens / taskCost / task

Earlier tests

Before the 30 tasks: 23 PowerWorld questions, answered with the kit and by the same model searching the web, marked blind. Then the case-auditor agent's own blind-judged test.

TestWith the kitWithout
Calls that report success and return something wrong15 of 167 of 16web search
Looking up a command name7 of 76 of 7web search
All 23 questions22 of 2313 of 23web search
The case-auditor agent described the case correctlyblind judge; build after v0.4.0, Sonnet21 of 2410 of 24plain model

Web search finds what a command does. It rarely finds that the call reports success and does nothing: on those 16 questions the web got 7 and the kit got 15. The kit also needed 3.2x fewer searches per right answer.

The case-auditor agent and a plain model got the same audit findings for 8 cases, 3 runs each. A blind judge found the agent described the case correctly in 21 of 24 runs, the plain model in 10. The plain model hardly ever invents anything: on one combined check (no invented cost data, only the three severity labels, no pasted raw findings) it passed 50 of 51, the agent 51 of 51. The difference is in the call: what actually stops the study.

Why the agent searches first: on 16 test questions, searching the pages found the right one 10 times, against 8 for the routing table and 7 for starting at the index. The full search rules

For exact argument syntax, open PowerWorld's manual instead. The kit deliberately does not publish it, so it sometimes names the right command without a signature. Every question, both verdicts, and what each setup spent

Wanna try it yourself?

Install HiveMind >_

One sentence pasted into Claude Code or Codex, then try it on your own case.

← Back to the benchmark

The 30 tasks

Each one is a request typed into a fresh Claude with HiveMind installed, with the case file on disk and nothing else. Click a task for the request as typed and the answer.

TOKEN MIXcached: re-reading its own contextnew inputoutput it wroteBars share one scale, so a longer bar is a task that took more tokens.