#Data/Training

AIThe post argues that China's commercial-scale EUV machines should be forecast for the late 2030s and immersion DUV for the mid-2030s, using ASML's development timeline as a reference. It also weighs factors that could push these estimates earlier or later, including state funding, espionage, talent flows, and the use of AI in R&D. The authors note that forecasts placing either milestone in the 2020s would need strong justification.

AIOpenAI reports that reinforcement learning on realistic conversations targeting traits such as honesty, epistemic humility, and corrigibility improved a model across 44 out-of-distribution alignment evaluations. Gains included reward hacking, deception, and health benchmarks, and training only on health conversations still improved non-health alignment scores. The trained model was also harder to steer toward harmful behavior with adversarial persona prompts or harmful fine-tuning.

Why it matters: The post tests whether reinforcement learning on beneficial traits in one domain transfers to unrelated alignment benchmarks and holds up under adversarial steering.

AIJohn Schulman says PPO gained a second wave in the LLM era for reasons not anticipated in the original paper. He points to the importance-ratio objective, which corrects biases from numeric error, asynchronous training, and forward-pass noise, and to the clipping objective, whose effect on entropy was unknown at publication, citing DAPO's arXiv paper.

AIOpenAI's alignment team found that re-generating 100,000 WildChat conversations with five recent OpenAI models predicted production failure rates across four orders of magnitude, with 95% of predictions within 1.04 orders of magnitude. The approach was weaker for agentic misalignment categories, where errors were about 37 times larger, and it still held roughly without access to chain-of-thought reasoning, with mean multiplicative error rising from 3.6x to 4.0x.

Why it matters: The post tests whether public chat data can predict real production failure rates, and where that prediction breaks down for agentic behavior.

AIMistral is working with companies and governments to keep their AI systems running outside external control and improving with each model release. Its Forge product enables continuous training of models based on recorded human-AI interactions, which the post calls a key unlock for efficiency.

AIMistral says it has moved its Studio deployment and Forge training products onto infrastructure it controls, decoupled from US service providers. Customers can run them in their own VPC or datacenter, or on Mistral's hosted capacity, which the company says is online and growing fast.

AIByteDance open-sourced inference code and model weights for Bernini-Diffusers, a full video generation and editing pipeline with an MLLM-based semantic planner and a DiT-based renderer. The release bundles a Qwen2.5-VL planner and Wan2.2 diffusion components in one self-contained directory, and the source recommends it over the renderer-only Bernini-R for complex instruction following.

AIXiaomi MiMo and the TileRT team say a 1T model exceeds 1,000 tps on a single standard 8-GPU node using general-purpose GPUs. The speedup comes from FP4 quantization and DFlash, a block-masked parallel speculative decoding method that accepts more tokens per verification, with TileRT tailoring its compiler and kernels to these techniques. Open weights for the FP4 + DFlash checkpoint are available on Hugging Face.

AISebastian Raschka has published a curated list of LLM research papers he bookmarked from January through May 2026, not a complete survey of the field. The list is weighted toward reasoning models, reinforcement learning, and efficient inference, with added interest in agent harnesses, long context, and diffusion language models. He highlights Nvidia's Nemotron 3 Super, a 120B-A12B hybrid model alternating attention and Mamba-2 layers, as a must-read, and notes a 4B Nano variant and the 550B-A55B Nemotron 3 Ultra released two days earlier.

AIFei-Fei Li praised GPIC, a new benchmark dataset for visual generation built for modern large-scale generative models. The corpus includes 100M VLM-captioned image-text pairs for training and 1M pairs for benchmarking, totaling about 28 trillion pixels. It is centrally hosted and fully permissive for research and commercial use.

AIEthan Mollick argues that using AI as a default for writing, without thinking, risks undermining the human effort that builds skill and style. He cites two Wharton-linked studies: a Turkish high school experiment where ChatGPT access hurt test performance, and a Taipei Python course where a personalized AI tutor raised exam scores by 0.15 standard deviations. Mollick calls the difference how AI is used, not whether, and notes that the tools for tutor-style learning are not intuitive to access.

AIMiniMax says its M2 series could not output the name Ma Jiaqi, a failure it traced to post-training data that rarely included the token. Its tests found the input embedding stayed stable while the lm_head weights for low-frequency tokens drifted during SFT. A synthetic full-vocabulary repetition dataset restored generation for affected tokens and reduced Japanese-to-Russian confusion from 47% to 1%.

Why it matters: The post traces a community-noticed token failure through tokenizer, embedding, and lm_head tests, showing how post-training data coverage can cause low-frequency token drift.

AITri Dao says that after a mathematical rewrite, all transformer operations can be expressed as a series of GEMMs with epilogues. Given a few optimized primitives, LLMs and novice humans can write near speed-of-light kernels for transformer ops. The related CODA work fuses memory-bound surrounding ops into the matmul epilogue, and LLMs can also write CODA kernels approaching speed-of-light.

AIStability AI released Stable Audio 3.0, a family of four audio models trained on fully licensed data. Three of them, Small SFX, Small and Medium, have open weights on Hugging Face, while Large is available through the Stability AI API and enterprise self-hosting. Outputs can be distributed and commercialized under the Stability AI Community License, and organizations with more than $1M in annual revenue can use the Enterprise License.

Why it matters: The source specifies which models are open-weight, their licensing terms, and clip-length limits, which matters for anyone deciding whether to build on them.

AISebastian Raschka reviews recent open-weight LLM architecture changes aimed at reducing long-context memory and compute costs. He covers KV sharing and per-layer embeddings in Gemma 4, per-layer query-head budgeting in Laguna XS.2, Compressed Convolutional Attention in ZAYA1-8B, and mHC with CSA/HCA compressed attention in DeepSeek V4. The article reports that DeepSeek V4-Pro uses 27% of single-token inference FLOPs and 10% of the KV cache size of DeepSeek V3.2 at a 1M-token context.

AIOpenAI's Fidji Simo argues that biological data infrastructure is the missing link needed for real-world impact, even though it lacks flashy announcements. She credits CZI (Chan Zuckerberg Initiative) for its work, pointing to the Virtual Biology Initiative linked in her post.

AIApple has released CADD-Base-7B on Hugging Face, a 7B masked diffusion language model for code generation that uses Continuously Augmented Discrete Diffusion (CADD) to guide discrete denoising with a continuous flow-matching signal. The model loads through Transformers with trust_remote_code, and its diffusion_generate method supports CADD sampling modes "weighted" and "argmax" with alg options such as "entropy" and "maskgit_plus". The release builds on DiffuCoder and reuses Dream's modeling architecture and generation utilities.

AITri Dao points to Wentao Guo's explanation of a mathematical rewrite of the MoE backward pass, which reduces activation memory and speeds up training, especially for fine-grained MoE. He also notes the work leverages new Blackwell features, including 2CTA MMA and CLC, to build fast MoE kernels.

AIResearchers introduced a simple, sample-efficient online reinforcement learning technique for post-training image generation models. It is presented as a possible steerable alternative to classifier-free guidance (CFG) that can be driven by any scalar reward, including human preference.

AIBerkeley AI Research introduces GRASP, a gradient-based planner for learned world models that aims to make long-horizon planning more robust. GRASP lifts trajectories into virtual states for parallel optimization across time, adds stochasticity to state iterates for exploration, and reshapes gradients to avoid brittle state-input gradients through high-dimensional vision models. The post identifies ill-conditioned gradients and non-greedy loss landscapes as core failure modes of standard rollout-based planning.

AIA developer trained a 12M-parameter LLM on a custom ML framework built with a Rust backend and CUDA kernels, including Flash Attention, fused LayerNorm, and fused GELU. The framework claims 3x throughput gains, WebGPU fallback for non-NVIDIA devices, and a TypeScript API installable via npm.

AIOpenAI released Privacy Filter, a bidirectional token-classification model that detects and masks personally identifiable information in text under the Apache 2.0 license. The model has 1.5B total parameters with 50M active, supports a 128,000-token context window, and can run in a web browser or on a laptop. Users can fine-tune it and adjust precision/recall tradeoffs through preset operating points.

AITri Dao says the dynamical system view yields very clean conditions for looped transformers to be stable. The post's background links looping blocks of layers during training to predictable scaling laws that can match a Transformer twice the size at a fixed parameter count.

AIJan Leike frames scalable oversight progress around weak-to-strong generalization, asking what fraction of a strong model's performance trained on golden data can be recovered using only a weak supervision signal. He points to an OpenAI blog post explaining the research.

AIARC Prize Foundation released an open-source human dataset for ARC-AGI-3, covering 342 step-by-step replays across 25 public environments from a study of 458 participants. The source reports that every environment was solved by at least two humans, and it updates scoring by moving the per-level baseline to the median human player and raising the per-level cap from 100% to 115%.

AICognition and Applied Compute RL-trained SWE-check, a specialized bug detection model for the Windsurf IDE. It matches frontier performance on in-distribution evals and is an order of magnitude faster with cheaper inference, though it trails frontier models on out-of-distribution evals (delta F1 0.29 versus 0.49 before training). A preview is available in Windsurf Next, with a mainstream release planned.

Why it matters: The post explains how production environment replication, reward linearization, and two-phase post-training trade bug-detection quality against latency for an IDE specialist model.

AIWerner Vogels announced S3 Files, which lets users mount any S3 bucket as a filesystem without making copies, running sync scripts, or choosing between file and object storage. He linked to a detailed post by Andy Warfield on the feature and its design history, including the filerectories concept that did not make the final release.

AITri Dao says a fast Muon optimizer is coming to consumer cards, since its symmetric matmul kernels work once Blackwell consumer GPU mainloop support is in place. Background from @jcz42 reports Gram Newton-Schulz symmetric kernels now support RTX 5090, with 2x faster Newton-Schulz and 1.7x faster optimizer time on 15 layers of Gemma-4 E2B.

AIAndrej Karpathy highlights Farzapedia, a personal wiki Farza built from 2,500 diary entries, Apple Notes, and iMessage conversations, as a good example of his proposed LLM-wiki approach. He argues this file-based memory is explicit, user-owned, interoperable, and usable with any AI tool, letting users control how AI knows them.

AIAwni Hannun says a 350M-parameter model trained on 28T tokens defies Chinchilla's compute-optimal scaling guidance. The quoted Liquid AI post credits scaled RL for LFM2.5-350M, reporting instruction following rising from 18.20 to 40.69, data extraction from 11.67 to 32.45, and tool use from 22.95 to 44.11 over LFM2-350M.

AIHamel Husain argues data scientists remain essential as foundation-model APIs let teams ship AI without them, because much of the work lies in evaluation, debugging, and metric design. He says teams often rely on generic off-the-shelf metrics and unverified LLM judges instead of examining their own data. He lists five eval pitfalls, starting with generic metrics, and recommends looking at traces and doing error analysis.

AIJim Fan argues that behavior cloning directly from humans, following EgoScale and its dexterity scaling law, has become the way to move past teleoperation. He says 2026 will focus on scaling robot learning without robots. The post is cited alongside EgoVerse, an ecosystem for egocentric human data with 1300+ hours across 240 scenes and 2000+ tasks.

AIHex CEO Barry McCardel posted a graph showing AI agents now create more Hex cells than humans do. Mintlify launched an analytics feature in February to track AI agent traffic to documentation, saying agents may read docs more often than humans. The article argues that writers should consider AI systems as a primary audience.

AIPrismAudio is a framework that integrates reinforcement learning into video-to-audio generation, using a Chain-of-Thought planning mechanism. It builds on ThinkSound by splitting single-step reasoning into four CoT modules for semantic, temporal, aesthetic, and spatial dimensions, each with targeted reward functions. Code, model weights, and datasets are released for research and educational use under the MIT License, and commercial use requires explicit author authorization.

AICursor says its Composer 2 anniversary marks one year of large model training, with a team of about 40 researchers and engineers now dedicated entirely to software engineering. The company states that every FLOP, token, parameter, and researcher is focused on coding rather than general assistant tasks. Composer 2 is now available in Cursor.

AITri Dao says nonlinear RNNs seem to do something genuinely different from attention and linear RNNs or SSMs. He reports they already perform well with the right parametrization, and adding just one nonlinear RNN layer substantially improves a transformer-Mamba/DeltaNet hybrid. The post quotes the M²RNN paper, which introduces non-linear RNNs with matrix-valued states for language modeling, with links to the paper, code, and models.

AIApple has published SimpleSD-4B-thinking on Hugging Face, a research checkpoint built on Qwen that improves code generation through Simple Self-Distillation without rewards, verifiers, teacher models, or reinforcement learning. On LiveCodeBench, it lifts Qwen3-4B-Thinking-2507 from 54.5% to 57.8% pass@1 on LCBv6 and from 59.6% to 63.1% pass@1 on LCBv5. The model is released as a reproducibility checkpoint under the Apple Machine Learning Research Model License, not as an optimized Qwen release.

AIApple has released SimpleSD-4B-instruct on Hugging Face, a research checkpoint fine-tuned from Qwen3-4B-Instruct-2507 on its own sampled outputs to improve code generation. On LiveCodeBench, the model scores 41.5% pass@1 on LCBv6, up from the base model's 34.0%, and 45.7% pass@1 on LCBv5, up from 34.3%. The model is released under the Apple Machine Learning Research Model License and is intended for reproducibility rather than as an optimized Qwen release.