#Data/Training

AIIn Zyphra's experiments, models with smaller context windows predicted text better while using less training compute. Zyphra believes the stronger recency bias induced by smaller windows outweighs the reduced compute benefit.

AIHybrid NoPE models combine sliding window attention or recurrent layers, which focus on nearby words, with global attention layers that use no positional encoding (NoPE). The post notes that NoPE layers receive no positional information yet can still learn long-range dependencies, and raises the question of how this works.

AILlamaIndex introduced Extract v2.5, a series of agents tuned for document extraction across cost-effective, agentic, and agentic plus tiers. The company reports the agents outperform Opus 5.5 and GPT-6 Sol while costing 30% to 4x less, with accuracy gains on long lists (86.1% to 95.5%), multi-page records (85.5% to 96.5%), and scanned forms (90.9% to 95.7%) on its agentic tier. The release adds advanced citations with bounding boxes and structural reasoning, and the agents are available on LlamaParse.

AIPerplexity Computer can now generate interactive charts and visualizations directly within a user's thread. For financial data, it uses TradingView Lightweight Charts to display candlesticks, volume, and moving averages.

AIHugging Face says ml-intern is an open-source ML engineering and research harness usable free on local setups, and it is also hosted on Hugging Chat with no-code access. A second hosted option runs on Hugging Face infrastructure, where ml-intern selects the cheapest GPU for a task so models can be trained for a few dollars. MaziyarPanahi reportedly trained a model by prompting alone for $6.60 on an NVIDIA A100 in 16 minutes.

AIThe SYNTH paper, titled It's All Training, presents a fully synthetic single-stage pipeline for training workable reasoning models with high data efficiency. The authors argue this approach does not separate training into pretraining, mid-training, or post-training stages. The image shows the paper's abstract, which describes a pipeline built from a 58,000-article Wikipedia-based synthetic corpus and models named Baguettotron-600M and Baguettotron-MoE.

AIDatabricks has introduced ai_decide, a new AI Function for fast, structured decisions over governed data. It classifies, scores, and chooses next actions in a fraction of a second, with lower latency and cost than an LLM on similar tasks. It is suited to model routing, document processing, agent evaluations, and real-time app logic.

AIHugging Face's Merve Noyan promoted a four-chapter YouTube series teaching viewers how to train AI agents using only open-source tools. The post links to the series' first video and is a brief call to learn agentic training.

AISara Hooker shared a full technical report on a method for creating datasets from zero seed data, linked via alphaXiv. She also pointed readers to an interactive tool at adaptionlabs.ai where users can begin inventing datasets.

AICloudflare's AI Search is now generally available, adding native image embeddings for visual retrieval, OCR for scanned PDFs, and a 10 MiB file limit up from 4 MiB. Billing begins November 1, 2026, charged per ingested token, stored GB-month, and query, with a free monthly allotment on all Workers plans.

AICohere's Sara Hooker announced the release of Invent a Dataset, along with a technical report addressing how to build a training set for a new capability with zero starting data. The work targets post-training a model when no data exists, a hard setting for frontier AI development.

AIThe Sequence argues that AI compute could become a commodity that requires a futures market, because unused GPU-hours cannot be stored and suppliers and buyers face forward-price risk. The piece says realizing this requires defining what is traded, measuring its quality, and building contracts around its risks, drawing on commodity market history.

AIAi2 released Olmo-core 3, an open training framework redesigned to scale mixture-of-experts models into the trillion-parameter range. In one benchmark, expert count rose from 8 to 128 with about 3.2B active parameters per token, total capacity grew from 4.6B to 47B, and throughput fell by less than 5%. The framework is fully open, so researchers can train their own MoEs and experiment with routing and parallelism.

Why it matters: The release documents concrete MoE scaling results and reported failure modes, useful for teams weighing training-stack tradeoffs before adopting an open framework.

AIIn a YouTube video, PewDiePie reportedly describes agentic training using GRPO, though the post says he does not name a specific tool. The author notes he attempted to distill Sol, was banned several times, and built a website for donated traces that received none, suggesting he could have used the Hugging Face Hub instead.

AIPhysicist Matthew Schwartz describes building BootLoops, an open-source harness for exact quantitative calculations, after choosing problems suited to Claude's strengths. He reports that Claude solved long-standing integrals and found connections across ecology, population genetics, economics, and linguistics, with domain experts steering results toward questions those fields care about. The post states that the approach required constant human oversight, since Claude often overstated results and misjudged time.

Why it matters: The guest post explains why scientists often find current AI tools frustrating and offers a method for finding problems where AI and researchers match, backed by concrete projects.

AIMastra platform now lets users attach a VPC-isolated Postgres database to any environment, restricting access to resources on the same network. The database cannot be reached from outside the network, so psql connections from external clients return an error. VPC Postgres joins Turso and Neon as managed database options, with MongoDB and Redis coming soon; it requires mastra@1.32.0 or later.

AIAmazon Aurora PostgreSQL can now query live database data and historical data in S3 within a single query, without copying data or maintaining sync pipelines. It uses DuckDB to read open formats such as Parquet and Iceberg directly where they are stored in the data lake.

AIFulcrum's Echo was trained using Fireworks' serverless training API on a rank 32 LoRA of Kimi K3. The team's writeup describes a "persona post-training" approach and a total training cost of $4.5k.

AIApple researchers introduced RLTL;DR, a reinforcement learning method in which an agent writes its own one-line insight after each failed attempt and learns to map tasks to those insights. On challenging tool-calling and coding datasets filtered to Pass@128 = 0, standard GRPO training of a Qwen 3.5 9B Thinking policy stayed at 0% to 1% Pass@1, while RLTL;DR reached 14–31% with insights in context and 12–13% without them at evaluation. A compact variant, SFTL;DR, trained on just 4k task-insight tuples recovered nearly the full performance of RLTL;DR.

AIMatei Zaharia announced that users can run decision models at scale within SQL and Spark queries. The post's quoted background describes Databricks' ai_decide function, which turns text into structured decisions over governed data for batch and REST API use.

AIPerplexity has published the pplx-embed-v2-context-9b-preview model on Hugging Face. The post links to the model page but gives no further details on its capabilities, benchmarks, or pricing.

AIPerplexity overcomes gold-chunk supervision limits by distilling relevance from its query-aware context compression model. The model scores every document token against the query, and those scores, aggregated into chunk-level targets, train the embedder to retrieve answer and supporting chunks.

AIRetrieval systems that split long documents into chunks lose surrounding context. Contextual embedding models address this by encoding the entire document once and pooling chunk vectors afterward. They are usually trained using one gold chunk per query.

AILlamaIndex held Tuesday-night events in New York and San Francisco on document processing for AI agents, with the New York room filling a waitlist and San Francisco drawing almost 600 attendees. The talks focused on the problem that agents often receive document text without its layout, so they must guess which figures, such as a monthly rate versus a total on an invoice, mean what.

AIGoogle DeepMind reports that AI-designed proteins can be synthesized and watermarked using its new SynthID Bio method, published in Nature. The team says the work is a step toward biosecurity in AI-driven biology and is open-sourcing the SynthID Bio tools for the research community.

AIHugging Face and OS4Science are partnering to identify the software libraries that scientific model contributors depend on most and to support their maintainers. The move responds to ESM-2, a 2022 protein model still downloaded hundreds of thousands of times a month, which relies on maintained software underneath it.

AIAgentic data science has AI agents explore datasets, choose modeling approaches, run analyses, and explain findings while data scientists frame questions and verify evidence. In an experiment, Claude Opus 5.0 given the vague prompt "Build me a model to detect fraudulent nodes" on a modified Elliptic Bitcoin dataset reported F1 0.87 and ROC AUC 0.99 using a random split that leaked a planted label proxy.

AIMicrosoft Research has developed a machine learning system that predicts where extreme space-weather events are likely to damage power systems 30-60 minutes before a storm arrives. Such storms can also degrade GPS accuracy and satellite operations, so advance warning could help operators prepare.

AIMicrosoft Research intern-developed machine learning pipeline forecasts location-specific geomagnetic risk for 66,935 substations in the continental United States. It combines solar-wind observations, AE and Dst forecasts, geological conductivity and grid data to estimate risk 30 to 60 minutes ahead. The pipeline detected nearly 80% of major space-weather events during the evaluation period.

AILiquid AI and InSilicoMeds released LongevityBench, an aging benchmark with 17 tasks spanning clinical records, DNA methylation, transcriptomics, proteomics, and genetics. On several tasks, Liquid AI's compact LFMs outperformed every frontier model the team evaluated. The team plans to present the work to the longevity research community at ARDD this week.

AIMicrosoft says its Circular Centers now operate eight facilities across North America, Europe, and Asia Pacific to decide the next life of decommissioned Azure hardware. Last year, Microsoft achieved a 92% reuse and recycling rate for decommissioned servers and components. Since 2014, Azure cores per rack have increased about 13-fold while power for the same task fell roughly 90%.

AIOpenBMB introduces Diffusion Reward Models (DRM), which learn the full reward distribution of human preferences instead of collapsing them into one scalar score. The approach preserves disagreement and uncertainty, enabling distribution-aware Best-of-N ranking and a new test-time scaling axis by sampling more reward outputs. DRM also improves downstream policy performance over scalar reward baselines when used as the reward in RLHF, according to the post.

AIDeepSeek has released an Ascend toolkit that mirrors its Nvidia components, including TileLang, DeepGEMM, DeepEP, TileKernels, FlashMLA and DeepSelect. It says every TileLang kernel used in its training now has a high-performance Ascend implementation. The post also reports that a 128-card Ascend 950 supernode, jointly optimized with Huawei, has key compute and communication tests approaching hardware limits.

AIJerry Liu says Jev, a System One model, outperformed other open-source classifiers and document-specific models on orientation detection, language detection, classification, and splitting. The benchmark measured accuracy, cost, and latency across these fast document decisions, with Jev leading most comparisons. The benchmark code is available in the run-llama/jev_vs_oss repository.

AIJerry Liu benchmarked gpt-6.1 sol on document OCR tasks and found a sizable increase in table parsing and reading order over gpt-6 sol from a week earlier. Its table parsing is similar to gpt-6 astra. He noted frontier models still cost roughly an order of magnitude more than cost-effective document parsing solutions, leaving room to improve the premium end above 1c per page.

AINumerical differences between a rollout engine and a trainer can make reinforcement learning collapse even when algorithm and data stay identical. In a GLM 5.2 experiment, reward fell from about 0.9 to under 0.2 around step 20 without alignment, while aligned numerics kept reward stable over 25 steps. A Qwen3.5-MoE investigation traced a significant mismatch to how expert outputs were combined, and router replay alone was judged insufficient.

AIMicrosoft's Azure AI team argues that better models do not eliminate the need for a dedicated content extraction layer, since agents need trustworthy, structured, and auditable inputs. The post notes that building extraction directly on an LLM quickly demands chunking, layout parsing, grounding, normalization, and evaluation infrastructure. Microsoft positions Azure Document Intelligence and Azure Content Understanding in Foundry Tools as managed options for that layer.

AIwh announced joining Proximal in San Francisco, citing open problems in data and post-training for large models. Proximal builds infrastructure that lets AI improve from real-world experience, and the background describes over $200M in annualized revenue from helping frontier labs and enterprises improve coding agents.

AIMicrosoft has made SQL Server on Azure Local generally available for connected and disconnected deployments, letting organizations run SQL Server in their own datacenters and edge locations. Disconnected operations continue locally where external connectivity is restricted or unavailable. Eligible existing SQL Server licenses can be used, and Foundry Local on Azure Local, currently in preview, brings AI inference alongside SQL Server data.

AIMicrosoft Research has introduced Quine, an early-stage research effort to build a multimodal world model of biology that connects insights across biological scales and modalities. The system is designed to help scientists computationally search a space far larger than intuition allows and prioritize hypotheses before lab testing. Experimental results are meant to feed back into the model and sharpen future research directions.