#Eval/Benchmark

AIArtificial Analysis reports that Grok Imagine Video 1.5 Lite comes closest to the frontier in Architecture & Real Estate, Consumer, and Productivity & Knowledge Work use cases. It sits furthest from the frontier in Live-Action Film and Frontier use cases. Against Grok Imagine Video 1.5, Lite matches it in Social Media & Creator Content and trails it on the other nine use cases.

AIArtificial Analysis shares an AA-Video-T2V v2.0 text-to-video prompt, a 10-second product-page scene of a presenter revealing a volcano-shaped mist diffuser. The prompt specifies a shift from warm daylight to dim evening lamplight, fixed close-up camera angles, and consistent presenter and product appearance across shots.

AIArtificial Analysis reports that Grok Imagine Video 1.5 Lite comes closest to the frontier on AA-Video-T2V v2.0 in Multi-Scene & Narrative, Lighting & Materials, and Text Rendering. It is furthest behind in Dialogue & Lip Sync and Human Anatomy. Compared with Grok Imagine Video 1.5, Lite matches it in Physics and trails on the other nine capabilities, by the least in Multi-Scene & Narrative.

AIAmong 12 models on AA-Video-T2V-Silent v2.0, Grok Imagine Video 1.5 Lite is the only one that is both fastest and highest quality, with no model beating it on both measures. It generates a 10-second 1080p clip in a median of 60.5 seconds. Kling 3.0 1080p (Pro) scores slightly higher but takes 94 seconds for a 5-second clip, while Vidu Q3 Turbo is 9 seconds faster on a 5-second 720p clip yet scores well below it.

AIGrok Imagine Video 1.5 Lite ranks #17 on AA-Video-T2V v2.0, two places above Google's Veo 3.1. At 1080p with audio, it costs $0.14 per second versus $0.40 per second for Veo 3.1. Compared with Grok Imagine Video 1.5, Lite is 44% cheaper at 1080p but ranks six places lower.

AISpaceXAI's Grok Imagine Video 1.5 Lite ranks #17 on both AA-Video-T2V v2.0 leaderboards, ahead of Google's Veo 3.1 at about a third of its price. It is the fastest model at its quality level in Artificial Analysis benchmarks, with a median of 60.5 seconds for a 10-second 1080p clip, and it costs $0.14 per second at 1080p, 56% of Grok Imagine Video 1.5's $0.25 per second.

AIEpoch AI reports that OpenAI halved GPT-6.1 Sol's cached-input price compared with GPT-6 Sol. Its measurements also show the model handles long prompts faster. The post raises, but does not confirm, a new architecture for GPT-6.1 Sol.

AISara Hooker announced a new blog post on adaptive checklists from Adaption AI, which she says capture current model capabilities and domain-specific requirements better than traditional checklists. She claims these adaptive checklists achieve this without requiring human supervision.

AIAnthropic's Claude Haiku 5.5 reached 1587 points on the Arena WebDev leaderboard, a 257-point gain over Claude Haiku 4.5's 1330. The post says it also improved significantly across all categories.

AIClaude Haiku 5.5 (High) debuted at #30 with 1587 points in Code Arena: WebDev, just outside the Pareto frontier. It matches GPT‑6 Luna's price at $0.10/$0.50 per 1M input/output tokens while scoring six points higher. Arena says it costs 90% less than Haiku 4.5 for +257 points, and 95% less than Sonnet 5.5 (High) for -128 points.

AIArtificial Analysis has formed the Cyber Index Alliance to set a new standard for evaluating how AI models perform on enterprise cyber defense tasks. Current members are Collinear, IBM, NVIDIA, and Vercel, and partners contribute expert input on the Index design and implementation, plus datasets and external research. Organizations interested in joining can contact cyber@artificialanalysis.ai.

AIGPT-6 Sol (Daybreak Blue, max) has been added to the Artificial Analysis Cyber Index as a trusted-access model and ranks #1 on the Index. Compared with the publicly available GPT-6 Sol, it shows its largest gains on CyberGym-E2E, the benchmark where the most safety refusals are observed.

AIGPT-6 Sol (Daybreak Blue, max) ranks first on the Artificial Analysis Cyber Index at a Cost per Task of $1.77. That is significantly cheaper than other leading models, including Grok 4.7 (xhigh) at $11.67 per task.

AIArtificial Analysis added trusted-access models to its Cyber Index, and GPT-6 Sol (Daybreak Blue, max) now leads the leaderboard. The model is available only through OpenAI's Daybreak program and records no safety blocks, improving 32 points over the publicly available GPT-6 Sol (max). It costs $1.77 per task, below Grok 4.7 (xhigh) at $11.67 per task.

AIGoogle proposed FlowAgent, a ReAct-style agent that generates and validates fixes for pre-submit test failures and shows them in its code review tools. Two abstention filters, before and after execution, suppress weak suggestions; in a manual review of 195 real failures, 67.18% of fixes were correct. After the Google-wide launch, it suggested fixes on 295,508 changes, with developers previewing 65,069 and applying 28,554.

AIGoodfire and PrimaMente report discovering a new class of Alzheimer's biomarkers. They are now opening a global AI competition to find new treatments, called the Alzheimer's Translation Challenge, led by PrimaMente and AlzData.

AIArtificial Analysis congratulated the launch of the ninth annual State of AI report, noting its benchmarks were featured throughout. The report, by Nathan Benaich, covers AI building better AI, world models, physical AI, geopolitics, cyber defense, and 12-month predictions.

AILiveKit is offering free access to its Simulations product through October, letting teams check what their agent can do and find gaps before deployment. The product also lets teams test any model against their own scenarios before switching models.

AIArtificial Analysis reports that generating more output tokens does not necessarily yield a higher score. GPT-6 Astra (max) scored 8.6% using about 81k output tokens per task, while Grok 4.7 (xhigh) used roughly 180k yet scored lower. Three Claude models produced the most output tokens, about 202k to 562k per task, but scored between 2.8% and 6.4%.

AIAmong models with a Hallucination-Gated All-Pass Rate above 0%, GPT-6 Luna (max), GPT-6.1 Sol (max), Muse Spark 1.3 (max), and Grok 4.7 (xhigh) set the Pareto frontier for score versus cost per task. GPT-6 Luna (max) is the cheapest at about $0.22 per task, scoring 3.3%, while Grok 4.7 (xhigh) leads at about $9.50 per task and Muse Spark 1.3 (max) costs about $4.20. The three Claude models cost about $18 to $22 per task.

AIOnce hallucinations are accounted for, Muse Spark 1.3 (max) drops from 26.7% to 8.9%, leaving Grok 4.7 (xhigh) first on the headline metric at 9.4%. Kimi K3 (max) falls from 16.7% to 5.3%, and Claude Sonnet 5.5 (max with fallback) falls from 11.7% to 2.8%. GPT-6.1 Sol (max) declines least, from 7.5% to 6.9%.

AIArtificial Analysis reports that Kimi K3 (max) achieves a 93.0% Criterion Pass Rate but averages 2.09 material hallucinations per task. Muse Spark 1.3 (max) scores higher at 96.0% while averaging 1.68 material hallucinations per task, showing that completing criteria and avoiding hallucinations are distinct skills.

AIArtificial Analysis compared six hallucination checkers on the same deliverables from 20 tasks across eight models. GPT-6 Sol and GPT-6 Luna generally flagged the most material hallucinations, while Claude Sonnet 5.5 and Gemini 3.8 Flash flagged far fewer, with Claude Opus 5.5 falling between Grok 4.7 and Sonnet. The counts reflect checker behavior rather than establishing accuracy or ruling out self-preference.