#Data/Training

AIByteDance Seed researchers found that language models compressing their KV cache in fixed-size chunks retrieve the same information unevenly depending on token position. In a 128K-token needle-in-a-haystack test, base DeepSeek-V4 checkpoints differed by up to 40.2 percentage points by phase, and post-training narrowed but did not eliminate the gaps. The authors urge evaluating such models across positional phases, since high average accuracy can hide systematic failures.

AISimplexity Robotics says it trained a single robot to load and unload two CNC lathes on its own, using 600 real-robot trajectories and reporting a 100% success rate on the precision CNC insertion task. The work, presented at IROS 2026 on September 29, combines the SimpleWAM world action model, a DRAM memory module and DPE action scoring, with force and torque feedback for insertion recovery. The company did not say how many trials the 100% figure covers, and it does not describe the yield of the whole cell.

AIMirroS released AgentGarten, which pairs executable code environments with a real-time neural renderer running above 30 fps so agents can act, observe, and learn. In a one-on-one hide-and-seek setup, the hider learned to block passages by round 4 and the seeker learned to climb ramps by round 10, guided by notes the agents wrote after each round. The authors report applying the same loop to four other tasks, including a dog-companion game, a narrow-bridge car passing task, herding, and quarry loading.

AIA Chinese-led team published in Cell a three-dimensional spatiotemporal cell atlas covering rice from germinating seed to grain fill, along with a public portal and the RICE scGPT single-cell foundation model. The atlas combines single-nucleus RNA sequencing with BGI's Stereo-seq spatial transcriptomics across 10 organ and tissue types and 61 stages, defining 119 cell types and 133 subtypes.

AISara Hooker congratulated @weiyinko_ml for leading a new work on automated data quality control with agentic checklists. She linked a blog post from Adaption Labs with further details.

AIAn astrophysicist worked with Claude Science to create the first complete ultraviolet map of the sky, covering regions never observed in UV. Claude located existing datasets, combined them, and filled gaps with statistical inference, taking a few days rather than weeks of human work. The map is presented as a teaching tool and an example of low-priority scientific work that AI now makes feasible.

AIEpoch AI estimates the AI chips shipped through 2027 could support about 30 to 170 million concurrent frontier-model agents, or nearly 2 billion with cheaper models. Its researchers find the cost of a fixed level of AI performance has fallen about 47% per quarter over the past three years. The newsletter also reports China's semiconductor supply-chain exposure is 2.7 times that of the US.

AIStability AI's Interactive Research team introduced SemanTok, which makes early video tokens more semantically meaningful so the representation is easier to predict. According to the post, a model using SemanTok matches or beats the performance of a model more than three times its size. The approach targets more efficient autoregressive video generation.

AIGoogle Research is presenting ContinuousBench, a standardized benchmark for measuring knowledge transfer and data contamination in differentially private (DP) synthetic data generation. Alex Bie is giving an encore presentation at the Google booth #107 at COLM 2026 today, at 2:00 PM PT.

AIThe Alzheimer's Translation Challenge is built on a new atlas of 150M cells, covering neurons, astrocytes, and microglia across different genetic backgrounds under combinatorial perturbations with multi-modal readouts. The data will be made available through the AD workbench and Prima Mente's modeling platform.

AIThe NVIDIA KGMON team placed second in the KDD Cup 2026 Data Agents competition with a system built around a smaller, clearer, and easier-to-verify agent harness. The competition required agents to answer natural-language questions over heterogeneous sources, including databases, CSV and JSON files, prose documents, PDFs, and briefing videos.

AI推荐阅读！ ExploreNet: Learning Where to Explore in Diffusion GRPO ExploreNet is a method introduced by @StellaLisy for learnable exploration in diffusion RL. It learns an adaptive exploration distribution conditioned on the current state, rewarded by the diversity of rollouts, aiming for faster, more targeted learning in diffusion GRPO.

AIRSIGym provides a research agent with training, inference, evals, and sandboxes as callable services, so it spends its budget on experiments rather than rebuilding infrastructure. With Opus 5 as the researcher, the improved system rose from 17.67% to 50.33% on SWE-bench Verified. The post also highlights a way to measure co-evolution between harnesses and models.

AIGoogle Research is presenting EnvHarness, a flexible framework that enables co-evolution between LLM agents and their training environments, at the #COLM2026 Google booth #107 today at 11:00 AM PT. The post notes that static environments limit agent growth, and EnvHarness is described as a plug-in architecture that dynamically reshapes environment behaviors to improve reinforcement learning and adaptability.

AIIn a prospective clinical study, Google's models pinpointed gestational age within 4 days of accuracy. The company says that precision could meaningfully affect clinical care, and that extending such tools to low-resource settings could help reduce maternal deaths and close care gaps worldwide.

AIZyphra has released a blog post and arXiv paper on expert coupling in mixture-of-experts (MoE) pretraining. The post itself provides only links, so the specific findings are not described here.

AIZyphra says its approach is lossless: the same tokens still reach the same experts, with unchanged architecture, routing decisions, and training objective. By reorganizing where experts and tokens live, it reduces the communication needed to perform the same computation.

AIZyphra reports that its MoE training optimizations speed up token exchange by 1.16x to 2.63x and full training steps by up to 1.41x in Megatron-LM on 8 to 64 GPUs. The gains are largest when each token uses more experts and those experts span several nodes.

AIZyphra reports that patterns in model training emerge early enough to be useful and are inexpensive to measure. Using only 4,096 tokens, its placement and routing predictions come within roughly one percentage point of results obtained with 524,000 tokens.

AIZyphra reports that expert routing in mixture-of-experts models is predictable across layers, since the experts a token uses in one layer indicate which it will need next. Its token shuffling method moves each token to the GPU holding those experts within a transfer that already runs after attention, adding no network traffic.

AIZyphra places experts that are frequently chosen together on the same GPU, so each token is sent to a GPU only once regardless of how many of its experts reside there. On 8 GPUs, this removes up to 58% of token copies sent across the network.

AIIn Zyphra's runs, MoE token exchange between experts consumed 13-24% of step time on one node and 45-60% across four nodes. Because experts are spread across GPUs and nodes, tokens must be sent to their experts and returned, making this communication a major training cost as models scale.

AIIn one layer, just 64 of 8,128 possible expert pairs handled 42% of tokens, far above the 1.6% expected if experts were chosen independently. Zyphra says this structure can be exploited to improve expert placement.

AIZyphra researchers report that exploiting patterns in how tokens are routed to experts makes Mixture of Experts communication up to 2.63x faster on AMD MI300X GPUs. The speedup comes without changing the model itself.

AILarge-scale AI training is spreading across multiple datacenters, with Google, Microsoft, AWS, Meta, and CoreWeave cited as examples. Because synchronized GPU clusters must exchange data in bursts, inter-site links can become a bottleneck, which Cisco estimates may require aggregate bandwidth about 14x a conventional DCI baseline.

AIReJev, an independent community project, applied LoRA post-training to OpenBMB's MiniCPM5-2B for bounded agent decisions: state, question, and candidate options yield one choice. On its sealed 1,892-sample holdout, accuracy rose from 51.11% to 80.50% (+29.39 percentage points) with 0% invalid outputs, at about $5.31 in cumulative Modal billing including earlier experimental overhead. The authors describe this as an early, task-specific result, not parity with Jev.

AINVIDIA, Princeton University, and the University of Maryland introduced PivotOPD, an on-policy distillation method that teaches multi-turn LLM agents to recover from their most damaging early mistake. Tested on Qwen3-1.7B and Qwen3-8B students, it posts the best average against 13 baselines on ALFWorld, WebShop, and Search-based QA. It recovers from 72.7% of replayed pivotal mistakes, versus 20.3% for standard OPD, with no added inference cost.

AIA Galbot, Tsinghua University and collaborators paper won IROS 2026's Best Entertainment and Amusement Paper Award for LATENT, a humanoid tennis-return method trained on imperfect amateur motion-capture clips. In simulation, the full forehand policy succeeded on 96.52 percent of returns, versus 71.85 percent for PULSE. On a real Unitree G1, the paper reports 90.90 percent forehand success across 20 consecutive rallies, with motion capture still used rather than the robot's own cameras.

AIHuawei engineers presented XMFS, an experimental Linux kernel prototype filesystem, at the Linux Plumbers Conference in Prague on October 5. It aims to let applications reach cross-node shared memory on CXL 3.0 or Huawei unified bus servers through standard POSIX file calls. The code exists only on openEuler, not in the mainline Linux kernel.

AIJohns Hopkins astrophysicist Brice Ménard, working as an Anthropic researcher, used Claude Science to produce the first complete map of the sky in ultraviolet light. Claude orchestrated agents to merge GALEX, Swift, and FIMS/SPEAR data, then predicted roughly a third of the sky that no UV telescope had observed, using relationships to visible, infrared, and radio data. Hidden test regions were reconstructed to within about 10% of real measurements, and each pixel is labeled measured or predicted with uncertainty estimates.

Why it matters: The post shows how an astrophysicist used Claude Science agents to merge UV surveys and predict missing sky regions, with a validation step that makes the method reusable.

AIApple researchers introduced Normalizing Trajectory Models (NTM), which model each reverse diffusion step as a conditional normalizing flow trained with exact likelihood. The model matches or outperforms strong image generation baselines on text-to-image benchmarks in just four sampling steps while retaining exact likelihood over the generative trajectory.

AIEpoch gave six models 11 real work tasks from its own operations, including graphic design, data insights, and research design, and graded outputs against employee standards. Fable 5.1 and GPT-6 Astra led on average task performance, reliably handling well-defined work such as coding and computational analysis. The report finds that all models still fail on open-ended judgment, including matching Epoch's standards, designing informative experiments, and generating diverse ideas, so the authors conclude AI cannot yet replace workers at Epoch.

Why it matters: The report separates well-defined task reliability from open-ended judgment failures, which benchmark scores on easily verifiable tasks would miss.

AIGoogle Research is hosting a walkthrough at its COLM booth #107 today at 5:00 PM of ContinuousBench, a standardized benchmark for measuring knowledge transfer in differentially private synthetic data. The session, led by Alex Bie, asks whether DP synthetic data preserve actual information or only style. A paper is linked on arXiv.

AIGoogle Research announced findings from a three-month randomized controlled trial with practicing patent attorneys, led by Visiting Fellow David Autor. The study examines both short-term productivity gains and longer-term skill building resulting from AI usage.

AINVIDIA researchers built PivotOPD, a training method that helps AI agents avoid early mistakes and recover when they occur. During training, a teacher model shows the agent a better action and guides it back on track over the next few steps.

AIGoogle Research is presenting EnvHarness at the #COLM2026 Google booth (#107) at 2:00 PM today, with Zifeng Wang leading the session. EnvHarness is a plug-in architecture that dynamically reshapes environment behaviors to improve reinforcement learning and agent adaptability, addressing limits of static training setups for LLM agents.

AIEpoch AI reports that AI models instructed to improve on several benchmarks could already be boosted by a recent human-authored post-training method, on-policy self-distillation (SDPO). The post implies these gains were known before the models' own technique was evaluated.

AIEpoch AI gave Fable 5 and GPT-5.6 Sol 3000 GPU-hours each to develop a new post-training technique intended to outperform the standard GRPO baseline. The post reports the task setup but does not state the results.

AIEpoch AI built InnovationEval to test whether AI agents can produce post-training innovations comparable in magnitude to a recently published advance. So far, the agents' results are underwhelming.

AIWorth reading — teach according to each student's aptitude! Sherpa: Teaching LLMs to Teach Adaptively