We planted 70 bugs in a 116k-line codebase. Across 3 runs, a single agent found 14, 15 and 27 bugs. A workflow consistently found 66 in each of its 3 runs.
Anthropic workflow finds 66 planted bugs; single agent finds 14 to 27
AISummary
Anthropic planted 70 bugs in a 116k-line codebase and tested two approaches over three runs. A single agent found 14, 15 and 27 bugs, while a workflow found 66 bugs in each run.
Post on XView on X
Source: ClaudeDevs · x.comPublished · added here