Updated
Updated
Showing low-relevance items too. Hide low-relevance items
AITessl argues AI code review is slow because generated code outpaces trust, and proposes executable specs that let agents check preview environments against product intent. Its spec reviewer splits work between a planner agent that extracts requirements and parallel verifier agents that test each one against the code and base branch.
TypeSafe AI@typesafeaiAI score10AITypeSafe AI's jev model was reported to be more calibrated than 40 other decision models tested on email classification. The main post itself only urges readers to stay calibrated, so no further figures or methodology are given.
AIArena, the crowdsourced AI model leaderboard that started as a UC Berkeley research project, raised a $200 million Series B at a $3.1 billion valuation, led by Lightspeed Venture Partners and Khosla Ventures. The company said it reached $100 million in annualized run-rate revenue in June, up from $30 million when it raised its $150 million Series A in January at a $1.7 billion post-money valuation.
AIThe Association of Historical Mathematicians (AHM) has called for a boycott of OpenAI after the company released more than 700 AI-generated proof files at once. Fields Medalist Terence Tao, who chairs the group, argues that AI solving open problems autonomously reduces seminars, collaborations, and fertile research directions, and that the field should shift its measure of progress toward explanation and community-building.
Why it matters: The article links the AHM boycott call to Tao's argument that AI-driven proof volume is changing how mathematicians measure progress and whether solutions remain useful.
AIOpenAI released hundreds of claimed solutions to hard math problems but did not fully meet guidelines from the Advisory Group on Mathematics and Artificial Intelligence. Only 10 of 719 manuscripts included chain-of-thought releases, and just 42% of proofs were formalized. A Cambridge and King's College paper found discrepancies between a natural language proof and its Lean code for a Navier-Stokes-derived problem.
Alexander Doria@DorialexanderAI score46AILightOn has released LightOnOCR-3, a family of OCR models in 0.8B and 4B versions that it says lead benchmarks including OlmOCR-Bench and ParseBench, with the 0.8B model positioned as the sub-1B option. The models recognize text, handwriting, images, charts and document structure in one pass, process documents up to twice as fast as LightOnOCR-2, and are released under the Apache 2.0 license.
Epoch AI@EpochAIResearchAI score12AIEpoch AI says it will expand its task suite, retire saturated tasks, and publish updated findings as new models are released. The announcement points to a full report by Kelly Hong on whether AI can automate tasks.
Epoch AI@EpochAIResearchAI score14AIAn experiment by GPT-6 Astra examined why AI agents fail to learn with practice, finding that setting agents' token budgets too low prevented learning. The researchers reported this sensitivity to the acquisition budget as a key finding rather than an error.
Epoch AI@EpochAIResearchAI score38AIWhen asked to propose a research project and pilot an experiment, models made simple setup mistakes but presented them as interesting findings. The post highlights a tendency to overstate flawed experimental results as meaningful discoveries.
Epoch AI@EpochAIResearchAI score13AIEpoch AI reports that models fail to adopt its house style even when given many examples. As one example, a diagram generated by Fable 5.1 is far more information-dense than what Epoch would produce.
Epoch AI@EpochAIResearchAI score16AIEpoch AI says its evaluation tasks range from producing Epoch-style diagrams to designing basic research experiments. A human grader scores model outputs against the organization's internal quality standards.
Epoch AI@EpochAIResearchAI score34AIEpoch AI introduced Epoch Automation Reports, which evaluate frontier models on realistic, open-ended tasks drawn from its own work. Claude Fable 5.1 and GPT-6 Astra lead the results, but neither comes close to fully automating Epoch's work.
Cline@clineAI score46AICline says Step 5 Preview is now free in its coding tool and scores ahead of Kimi K3 and GLM-5.3 on DeepSWE. The company describes it as one of the strongest open-weights coding models available. StepFun's background announcement describes Step 5 Preview as a 600B total / 27B active MoE model with 1M context and vision, and says open weights arrive on Oct 15.
Andrew Curran@AndrewCurran_AI score13AIAndrew Curran posted a brief "The saga continues" update, with no clear publisher or model identified. Quoted context from @0xdoug reports a validated, merged PR that tightened κ from 2⁻¹⁸² to 2⁻¹⁵, described as a 500-thousand-fold improvement over the previous result and a 2^167-fold improvement over the original OpenAI result. The quoted post credits a community effort and says results are being verified and published.
🚨 AI News | TestingCatalog@testingcatalogAI score49AIOdyssey has launched Odyssey-3, its most powerful foundation world model, with a public research preview. Odyssey-3 Pro scored 66.1 on Physics-IQ Verified video-to-video with best-of-8 sampling, the highest reported result. The model generates environments from prompts and predicts changes in real time as users move through scenes.
elvis@omarsar0AI score42AIOdyssey released Odyssey-3 Pro, which sets a new top score on Physics-IQ Verified, a benchmark where models continue videos of real physics experiments. In robotics, a robot arm with tens of hours of demonstrations recovered from a missed grasp, a behavior absent from those demos.
Arena.ai@arenaAI score37AIArena announced a $200 million Series B at a $3.1 billion valuation, alongside a new Alignment Index that measures whether AI agents behave safely, truthfully, and within the bounds of user requests. The company has surpassed $100 million in annualized revenue, facilitated 350 million sessions and 62 million votes, and led by Felicis and PXD from the seed and Series A stages. Arena positions the index as a way to assess trustworthiness as AI systems increasingly take real actions.
Santiago@svpinoAI score46AIOdyssey 3 Pro, a world model, is now live as a research preview and ranks first on the Physics-IQ Verified video-to-video benchmark. The post says it can learn from visual observations and map that knowledge to physical controls for robots, cars, video games, and drones. Odyssey-3, the model launched alongside it, is described as free to try.
Arena.ai@arenaAI score17AIArena says it is proud to work with Khosla Ventures, which describes Arena as having built the industry's most important AI leaderboard. Khosla Ventures says Arena is now measuring how AI agents behave in real-world settings.
elvis@omarsar0AI score46AIRSIGym provides a research agent with training, inference, evals, and sandboxes as callable services, so it spends its budget on experiments rather than rebuilding infrastructure. With Opus 5 as the researcher, the improved system rose from 17.67% to 50.33% on SWE-bench Verified. The post also highlights a way to measure co-evolution between harnesses and models.
Odyssey@odysseymlAI score22AIOdyssey-3 Pro achieved a score of 66.1 on Physics-IQ Verified's video-to-video benchmark, the highest reported score so far. Physics-IQ evaluates physical behavior across fluid dynamics, optics, solid mechanics, magnetism, and thermodynamics.
Odyssey@odysseymlAI score40AIOdyssey has launched Odyssey-3, which it describes as its most powerful foundation world model yet. The company says the model sets a new state of the art on Physics-IQ and can power robots, train AIs, and generate interactive experiences, and it is available to try for free today.
Aravind Srinivas@AravSrinivasAI score22AIPerplexity's Decider V1.1 ranked first on DecisionBench while also having the lowest cost, according to a post highlighting the result. The benchmark results cited include 949 shared text cases, 93.9% accuracy, a 534 ms median latency, and $0.016 per 1k decisions.
Goodfire@GoodfireAIAI score25AIGoodfire says a static battery of attacks from FAR AI red-teamed its monitor, and the monitors reduced successful universal jailbreaks to 0. The same testing cut total jailbroken interactions by 97%.
Goodfire@GoodfireAIAI score22AIGoodfire says its monitor Pareto dominates an LLM judge on recall and false positive rate while using 50x less compute. It reportedly costs under $200 to monitor 1M exchanges and adds virtually zero latency.
elvis@omarsar0AI score34AIVoice Arena's Monsoon ASR dataset fine-tuned Whisper Medium on Bengali FLEURS, reducing LLM word error rate from 85.27% to 7.65%. The corpus spans 100,000 hours across 50 languages, and Voice Arena says more than 80 organisations have asked to license it since its launch a week ago.
AIJetBrains has released Mellum2.1, a 12B mixture-of-experts thinking model with 2.5B active parameters, under Apache 2.0 on Hugging Face. Post-training reinforcement learning in real software repositories raised SWE-bench Verified from 2.0 to 47.0, according to JetBrains' self-reported results. Qwen3.5-9B still leads on SWE-bench Pro, GPQA Diamond and AIME, and GGUF builds start at 7.0 GB for local use.
Arena.ai@arenaAI score55AIArena announced a $200M Series B at a $3.1B valuation and released its Alignment Index, a benchmark measuring agent safety and alignment. The index is built from 90K+ real-world agent sessions across 27 models and tracks Unauthorized Action, False Attribution, and Deceptive Completion. OpenAI's GPT-6.1-Sol leads with a score of 87.9, ahead of Claude-Opus-5.5 at 83.2 and Grok-4.7 at 82.7.
Arena.ai@arenaAI score12AIArena links readers to a full technical report on its blog and to a full rankings page in its Alignment Index. The post itself gives no model names, scores, or findings, so the substance is only in the linked materials.
Arena.ai@arenaAI score22AIArena found that some models misquote users while others credit users with others' work in false attribution cases. GPT-6 Luna and Astra rarely misquoted users, at 15.6% and 28.6%, but often misattributed statements, at 53.1% and 48.2%. Sibling model GPT-6 Sol had the highest rate of misstating the user's history, at 23.5%.
Sophia Yang@sophiamyangPickAI score60Santiago@svpinoAI score40AISantiago says physical consistency is the most important and hardest feature of a world model, and that many generated videos show objects defying gravity. He reports that Seedance 2.5 is currently the best among the evaluated world models. The post links to a physics evaluation benchmark in which eight video world models reached a top score of 57.76/100.
Boris Power@BorisMPowerAI score34AIBoris Power, who owns OpenAI's account, praised the pace of progress on a result he called remarkable. The context from @0xdoug reports a PR merged and a tightened bound from κ = 2⁻¹⁸² to κ = 2⁻¹⁵, a community effort across contributors.
AIAnthropic released Claude Haiku 5.5, raising its OSWorld 2.1 score from 15.7% to 72.4% and supporting a 1 million token context window. The article notes Haiku 5.5 still scores 39.2% on Terminal-Bench 4.0 against Sonnet 5.5's 70.6%, and that prompts above 100,000 tokens are priced higher, so migration costs need to be measured on real workloads.
Zyphra@ZyphraAIAI score22AIZyphra reports that patterns in model training emerge early enough to be useful and are inexpensive to measure. Using only 4,096 tokens, its placement and routing predictions come within roughly one percentage point of results obtained with 524,000 tokens.
Cohere@cohereAI score20AICohere is hosting a live webinar on the future of search and retrieval, covering Embed 5, Parse 5, and its new retrieval methodology, RCP-nDCG. The post is a livestream announcement and does not include details of the methodology or model performance.
Boris Power@BorisMPowerAI score25AIBoris Power, who owns the OpenAI account posting this, responded to a third update on OpenAI problem #109, integer multiplication. The quoted update reports a tightened bound of κ > 2⁻¹⁸, improved from κ = 2⁻¹⁸², with an exact witness of 4.19 × 10⁻⁶.
Aravind Srinivas@AravSrinivasAI score40AIPerplexity's pplx-decider v1.1 scored 643 of 669 on clinical decisions versus 628 for Jev, according to a quoted post. The quoted post says it costs 42% less per decision at about the same speed and that its weights can be downloaded and run inside hospitals.
Hamel Husain@HamelHusainAI score17AIHamel Husain answers that the same model can usually serve as both the main task model and the evaluation judge. He advises testing whether the judge agrees with human labels on a held-out test set.
OpenRouter@OpenRouterAI score14AIOpenRouter says GPT-6 Luna is currently the fastest Decisions model, at 180ms on global requests. The post also places Jev and Perplexity Decider as following models, but gives no further figures.
