#Eval/Benchmark

AINew research finds AI tutoring delivers immediate GRE learning gains comparable to expert human tutors, with Gemma 4 31B performing best. The post presents the findings as a way for educators to scale their impact and make guidance more accessible.

AIGoodfire built monitors that let AI agents perform more biology work while flagging dual-use risks. The monitors reportedly outperform frontier model safeguards, with fewer refusals on dual-use tasks, and run fast and in real time while staying robust to adversarial attacks.

AIPrime Intellect reports that after about 100 GRPO steps on Hosted Training, Qwen3.6's reward on held-out problems rose from 0.127 to 0.361, a 2.8x gain. Qwen3.5, trained the same way, reached 0.356, suggesting the method works across model families. Both post-trained models finished well ahead of other open models and narrowed the gap to Claude Opus 4.8, with Qwen3.6 activating only 3B parameters per token.

AILlamaIndex introduced Extract v2.5, a series of agents tuned for document extraction across cost-effective, agentic, and agentic plus tiers. The company reports the agents outperform Opus 5.5 and GPT-6 Sol while costing 30% to 4x less, with accuracy gains on long lists (86.1% to 95.5%), multi-page records (85.5% to 96.5%), and scanned forms (90.9% to 95.7%) on its agentic tier. The release adds advanced citations with bounding boxes and structural reasoning, and the agents are available on LlamaParse.

AIHugging Face shows that training LFM2.5-2.6B with RL inside the agent harnesses themselves lifted held-out task success from 42% to 54% across four harnesses. Before training, the model solved 62% of tasks in Mini-SWE-Agent but only 33% in Claude Code, so the same model behaved very differently per harness. The approach uses an OpenEnv capture proxy to record tokens and logprobs, Harbor for tasks and sandboxes, and TRL's async GRPO trainer, with 31% fewer tool calls on already-solved tasks; training in OpenCode alone mostly improved OpenCode.

AIHamel Husain advises against building automated evaluators for every failure mode discovered during AI development. He argues that teams should weigh the costs of different evaluator types, such as code-based checks versus LLM judges, before building an eval.

AIHamel Husain has published a longer review of a new Claude Auto Eval plugin after many users asked about it. He invites readers to share their experiences using the plugin and how it went for them. The plugin is part of Claude's ability to help build evaluations and hillclimb on them, as described by @ClaudeDevs.

AIGoogle has announced Gemini 4 Argon, initially available only to trusted cyber defenders through its Fairwind Program while US government approval is pending. The author says the model is aimed at long-running software engineering, enterprise knowledge work, and cybersecurity tasks, with a 1 million token output limit. The post also gives promotional pricing of $2 per million input tokens and $10 per million output tokens, rising to $4 and $20 afterward, alongside a benchmark comparison.

Why it matters: The post places Gemini 4 Argon's benchmark table beside GPT-6 Astra and Claude models, showing where each leads across coding, knowledge work, and cybersecurity tasks.

AIVarun Mohan says Gemini 4 Argon is doing well on the AA Coding Agent Index when run with the Antigravity harness. He notes that real-world usage matters more than benchmarks and promises more to come.

AIThe CDC announced that Google's science AI model ranked highest among 39 eligible models for forecasting flu-related hospital admissions during the 2025-26 flu season. Google's forecasts were built with Empirical Research Assistance, an AI tool that generates computational solutions across scientific fields.

AIThe post quotes Google's Sundar Pichai introducing Gemini 4 Argon as an early look at the next model. It claims frontier performance in complex workflows, cyber defense, and software engineering, and says Google teams are using it for tasks from coding to quantum computing. The author adds that on FrontierSWE the model is very self-critical and often says "Eureka!", a personality they describe as a large improvement over previous Gemini models.

AIYuchen Jin suggests Google is back in the AI race after a model reportedly outperforms Astra and Opus 5.5 across the board on benchmarks. He adds a caveat that the results may reflect benchmark optimization rather than real capability gains, and says he would welcome Google rejoining the race.

AIPerplexity's embedding preview achieves the highest average nDCG@10 among tested models on ConTEB, though not on every task. It also outperforms voyage-context-4 on chunk retrieval while using 8x less storage per vector, at 1 KB (1024 dims, int8) versus 8 KB (2048 dims, float32).

AIPerplexity says its pplx-embed-v2-context-9b-preview model leads on Answer and Evidence retrieval at every cutoff. In document retrieval the gap is smaller, with its context v1 4B model slightly ahead at Document@3 and Document@5.

AIPerplexity says context-bench, a context-aware retrieval benchmark created and privately held by turbopuffer, has 2,099 queries and 38,894 documents. Perplexity submitted it for blind evaluation, with queries and capabilities inspired by turbopuffer customer conversations.

AIPerplexity introduced a new training method for contextual embedding models that encode each document chunk with the whole document in view. Its pplx-embed-v2-context-9b-preview sets a new state of the art on the ConTEB benchmark and on turbopuffer's privately held context-bench.

AITerminal test runs miss multi-turn agent regressions, so Agent Clinic Episode 3 builds an automated eval suite for a LangGraph agent in 60 minutes. The post presents a four-step framework for moving from informal checks to benchmarking AI agents, with a link to the full guide.

AINathan Lambert's latest Interconnects plot tracks daily tokens processed by leading open model inference providers. The figures come from public disclosures by Together, Baseten, and Fireworks, plus the OpenRouter API. He says he underestimated OpenRouter's growth.

AIAnt Ling reports that Ling-3.1-flash scored 65.35 on HealthBench Professional, a healthcare evaluation rather than clinical certification. The model was trained with feedback from AQ and Haodf.com (Good Doctor Online) healthcare scenarios.

AILing-3.1-flash achieved a 1.432x speedup on Pyright 1.1.400 while passing all 2,279 checks. The chart shows Claude Fable 5 at 1.502x and GLM-5.3-Flash at 1.290x, a close result on a real codebase with correctness intact.

AILing-3.1-flash built a Lua-to-x86-64 ELF compiler from scratch in about 17 hours, passing 178 of 182 tests for 97.8%. That is within 1.2 percentage points of the 99.0% shown for Claude Fable 5 and above GLM-5.3-Flash at 95.9%.

AILing-3.1-flash ranks #2 in the Mobile App (react native) category on Design Arena's Open Weights leaderboard, with 1,207 Elo. It also ranks #4 in SVG with 1,271 Elo among models added in the last 30 days.

AIAnt Ling introduced Ling-3.1-flash, a model with about 560B total parameters, about 25B active per token, and up to a 1M-token context window. The company plans to open-source the model soon. It reports 1,673 Elo on GDPVal-AA v2.1, 75.16 on FrontierSWE, and 65.35 on HealthBench Professional across work, coding, and healthcare tasks.