LangSmith's Jev judge scores agent trajectories on difficulty and correctness
Original titletrajectory labeling is really several questions, not one pass/fail - nice framing here
AISummary
Harrison Chase says trajectory labeling should be split into several questions rather than one pass/fail judgment. LangChain's Jev judge in LangSmith Evals answers difficulty and correctness as separate typed outputs in a single pass on every trace.
Source: Harrison Chase · x.comPublished · added here