Skip to content
View original post on X: Jerry Liu· 26/100AI score26/100

Jerry Liu says evals now replace hand-built agent workflows

AISummary

Jerry Liu argues that most tasks can now be solved by defining an eval and hillclimbing on it, rather than hand-coding a deterministic or agentic workflow.

He says data provider companies are building evals across economic activity so frontier models can handle more work, leaving developers to define goals and success measures.

He expects agent interfaces to compress most tasks into goals and eval instructions, while the most complex processes will still need explicit workflow builders.

Post on XView on X
@jerryjliu0

These days, you can pretty much solve any task by defining an eval and hillclimbing over it, instead of directly defining the deterministic/agentic workflow to solve it.

The data provider companies' entire job is to define evals for all economic activity to make the frontier models capable of doing anything.

Your job then becomes pointing frontier intelligence in the right direction - defining what to solve, and how to measure what good looks like.

Agent application interfaces will evolve to capture this. The most complex processes will still need some sort of explicit workflow builder interface, but most tasks can be compressed into goals and eval instructions.

Source: Jerry Liu · x.comPublished · added here