Skip to contentSkip to stories

Updated

#Multimodal

Showing low-relevance items too. Hide low-relevance items

Aug 4

Aug 4Tue

Aug 1

Aug 1Sat
  1. Andrej KarpathyAI score66

    Karpathy tests Opus 5 by rendering Lord of the Rings opening in 3D

    AIAndrej Karpathy gave Claude Opus 5 the first paragraph of Lord of the Rings with a 1M token budget and asked for a Three.js render. Opus spent about two hours writing 5500 lines of code that procedurally renders the story, which Karpathy calls janky but fun. He notes the model struggled to audit its work because it cannot efficiently perceive video or play the resulting game, relying on slow screenshots that led to several errors.

    Video from @karpathy's post

Jul 30

Jul 30Thu
  1. MiniMax BlogAI score72

    MiniMax H3 unifies text, image, video, and audio generation in one model

    AIMiniMax launches H3, a general-purpose multimodal generation model that understands text, images, video, and audio as unified context. It generates video up to 15 seconds at 2K resolution with native stereo sound, and the company says model weights will be opened in the coming days, subject to applicable laws and regulations. MiniMax also says H3 is priced below mainstream models at 2K and 768p.

    Why it matters: The post explains how a unified multimodal design and training choices enable 2K video with native stereo sound, useful for comparing against closed video generators.

  2. Soumith ChintalaAI score57

    Thinking Machines releases Inkling-Small, a 276B-parameter model with full weights

    AIThinking Machines is releasing Inkling-Small, which it says achieves performance comparable to Inkling at a quarter of its size. The model has 276B total parameters with 12B active, and the full weights are available. Users can fine-tune it on Tinker or chat with it in text, image, and audio on Tinker Playground.

Jul 29

Jul 29Wed
  1. Google LabsAI score60

    Google Launches Lyria 3.5 in Flow Music With Better Vocals and Lyrics

    AIGoogle is rolling out Lyria 3.5, its newest music generation model, in Google Flow Music today. The update improves musicality, lyric quality and prompt adherence, and vocal expressiveness and pronunciation, and gives users more control over tempo and duration.

    Why it matters: The post names the specific capability changes and where users can access them, which helps readers judge fit for music creation workflows.

  2. Alibaba NLP (Tongyi) · new models on Hugging FaceAI score40

    Alibaba NLP releases UEmbed-9B, a unified sparse and dense multimodal embedding model

    AIAlibaba NLP has released UEmbed-9B, a decoder-only multimodal embedding model built on Qwen3.5 9B that outputs both dense and SPLADE-style sparse embeddings from one forward pass. It supports text, image, video, and mixed-modal inputs for retrieval and multimodal search, and the family also includes 2B and 4B variants. The model is available on Hugging Face, with transformers and vLLM inference support.

  3. Alibaba NLP (Tongyi) · new models on Hugging FaceAI score38

    Alibaba NLP releases UEmbed-4B, a unified sparse and dense multimodal embedding model

    AIAlibaba NLP has released UEmbed-4B, a decoder-only multimodal embedding model built on Qwen3.5 4B that outputs both dense and sparse embeddings from one forward pass. It handles text, image, video, and mixed-modal inputs for retrieval and visual-document search, and sparse activations map to vocabulary terms usable with inverted indexes. The model is available on Hugging Face in a family that also includes 2B and 9B variants.

  4. Alibaba NLP (Tongyi) · new models on Hugging FaceAI score43

    Alibaba-NLP releases UEmbed-2B, a multimodal model producing dense and sparse embeddings

    AIAlibaba-NLP's UEmbed-2B, a decoder-only multimodal embedding model built on Qwen3.5 2B, produces both dense and SPLADE-style sparse embeddings from a single forward pass. It supports text, image, video, and mixed-modal inputs for retrieval, and the 4B and 9B variants are also available. The team reports state-of-the-art results on the text and agent tracks of MMEB-v3.

Jul 28

Jul 28Tue
  1. MiniMax · new models on Hugging FaceAI score76

    MiniMax H3 releases open-weight omni-modal video model with native stereo audio

    AIMiniMax released H3, an open-weights omni-modal model that generates video with native stereo audio up to 2K and 15 seconds. The system combines H3-Context-IR preprocessing, the H3-Base generator at 768p, and H3-Regenerate-2K for 2K output, with the Context-IR and 2K modules available only through API.

    Why it matters: The source details a three-module pipeline and open weights with deployment paths, showing how a video model is served and reproduced locally.

  2. Intern Large ModelsAI score62

    Intern Large Models introduces Visual Pretraining learned from visual documents

    AIIntern Large Models introduces Visual Pretraining, a pretraining paradigm for foundation models that learns directly from visual documents. The post says it outperforms text-only pretraining across backbones and benchmarks, and links the arXiv paper 2607.09657 along with Intern-S2-Preview (35B) and Intern-S2-Preview-397B on Hugging Face, the latter presented as a multimodal foundation model trained with this recipe.

    Image from @intern_lm's post

Jul 27

Jul 27Mon
  1. Kimi.aiAI score46

    Kimi releases PerceptionBench, a benchmark isolating 10 atomic visual perception capabilities

    AIMoonshot AI's Kimi has released PerceptionBench, a benchmark that evaluates visual perception as atomic capabilities derived from frontier-model failures across 42 benchmarks. It contains 3,000 verified questions, each isolating a single capability and answerable by looking alone, without reasoning or external knowledge.

    Image from @Kimi_Moonshot's post
  2. Kimi.aiAI score86

    Moonshot AI releases Kimi K3 weights and technical report

    AIMoonshot AI is releasing the model weights and technical report for Kimi K3, a 2.8T-parameter MoE model with native visual understanding and a 1M-token context window. The post says the new architecture delivers 2.5x the intelligence per unit of compute, and the company is also opening high-performance attention kernels, an MoE communication library, and infrastructure for running agent environments at scale.

    Why it matters: The source names the model size, context window, and released weights, which helps readers compare its scale and openness with other frontier releases.

    Image from @Kimi_Moonshot's post
  3. Air Street PressAI score72

    Black Forest Labs releases FLUX 3, extended to video and robot control

    AIBlack Forest Labs released FLUX 3, a multimodal model trained on images, video, and audio, and mimic built FLUX-mimic on its video backbone to control robots. In a soft-body kitting task, mimic reports a 95% success rate without single-task fine-tuning, compared with 55% for an adapted π0.5 model. FLUX 3 Video is in early access, with action prediction offered to selected partners and an open-weight backbone planned.

Jul 24

Jul 24Fri

Jul 23

Jul 23Thu
  1. One Useful Thing (Ethan Mollick)AI score67

    Ethan Mollick's guide to choosing AI tools for agentic work

    AIEthan Mollick's guide says ChatGPT and Claude are the main choices for real work, since their agent modes can act on a computer. He separates agent modes that run on the company's computers from those that access the user's own computer. He recommends keeping approval settings on for sending, spending, or deleting, because of prompt injection risk. He also notes that Gemini currently lags for agentic work, though its Notebook and video tools are useful.

Jul 22

Jul 22Wed

Jul 21

Jul 21Tue
  1. Andrej KarpathyAI score30

    Karpathy suggests long voice rambles help LLMs understand your intent

    AIAndrej Karpathy describes using /voice to ramble for about 10 minutes, sometimes as a short interview, to give an LLM context that would be tedious to type. He says LLMs reconstruct these messy streams of thought remarkably well, often returning a cleaner version than the speaker started with, which improves shared understanding and reduces later corrections.

  2. koray kavukcuogluAI score72

    Google releases Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber

    AIGoogle introduces Gemini 3.6 Flash as its workhorse model, with better coding, knowledge work, and multimodal performance while reducing token usage. It also launches Gemini 3.5 Flash-Lite, described as the fastest and most cost-effective 3.5-class model for high-throughput applications, and 3.5 Flash Cyber, a version of 3.5 Flash fine-tuned to find and fix cybersecurity vulnerabilities.

    Why it matters: The post lists three distinct models, each aimed at a different job, so readers can map which one fits coding, high-volume, or security workloads.

    Video from @koraykv's post
  3. Air Street PressAI score67

    DeepMind's Raia Hadsell argues AI should move beyond language to world models and robotics

    AIAt RAAIS, DeepMind VP of Research Raia Hadsell argued that the field focuses too much on language and should apply large-model training to worlds, robots, biology, and weather. The article cites DeepMind's DiffusionGemma, a 26-billion-parameter open text model that generates blocks by denoising rather than one token at a time, and the Genie-3 world model, which runs in real time for several minutes. It also describes world models as a source of synthetic training data for robots.

Jul 16

Jul 16Thu
  1. Soumith ChintalaAI score60

    Kimi K3 launches as a 2.8 trillion parameter open-weight model

    AIMoonshot AI announced Kimi K3, a native multimodal model with 2.8 trillion parameters and a 1 million token context window. The announcement cites up to 6.3x faster decoding in million-token contexts and about 25% higher training efficiency, and says open weights arrive by July 27, 2026. The author, Soumith Chintala, reposted it with a brief note of congratulations.

  2. Mistral AI · new models on Hugging FaceAI score46

    Mistral releases Shieldstral-1.0-3B, a policy-adaptive multimodal safety classifier

    AIMistral AI released Shieldstral-1.0-3B, a 3B-parameter multimodal safety classifier that judges content against natural-language policies and outputs a continuous safety score. It moderates text, image, and text-plus-image content in a single forward pass and can be retargeted to new policies at inference time without retraining. The Apache 2.0 open-weight model is built on Ministral-3-3B-Base-2512 and trained on sequences up to 32k tokens.

Jul 15

Jul 15Wed
  1. Leandro von WerraAI score42

    Thinking Machines releases Inkling, a multimodal model with open weights

    AIThinking Machines has introduced Inkling, a model that reasons across text, image, and audio, with full weights made available. It is available for fine-tuning on Tinker and can be tried in the Inkling Playground. Hugging Face's Leandro von Werra praised the release for its grounded writing, interesting details, and strong ecosystem integration.

  2. Lilian WengAI score62

    Thinking Machines releases Inkling, an open-weights multimodal model

    AIThinking Machines has introduced Inkling, an open-weights model that reasons across text, image, and audio, with full weights made available. The model is available today for fine-tuning on Tinker, and the company also offers an Inkling Playground for trying it out. The author describes Inkling as a foundation model intended for broad capabilities in practical use and customization.

  3. John SchulmanAI score75

    Thinking Machines releases open-weights multimodal model Inkling

    AIThinking Machines introduced Inkling, a model that reasons across text, image, and audio, and is making its full weights available. It is available today for fine-tuning on Tinker and can be tried in the Inkling Playground. John Schulman says pretraining began last winter and a small team added coding, reasoning, and agentic training starting in mid-January.

    Why it matters: The post links an open-weights release to a stated training timeline, showing how a small team moved from pretraining to coding, reasoning, and agentic training.

Jul 13

Jul 13Mon