#On-device

AIOllama announced that Google DeepMind's EmbeddingGemma 2 is now available on Ollama. The author describes it as made for consumer devices and multimodal, and gives the command ollama pull embeddinggemma-2 to download it. The quoted DeepMind post says the model is a natively multimodal open model for on-device embeddings that unifies code, images, audio, and video in a shared space.

AIGoogle DeepMind has released EmbeddingGemma 2, an open 740 million parameter model that maps text, images, audio, and video into one embedding space. It is built on the Gemma 4 architecture under an Apache 2.0 license and supports an 8K token context window. The company reports a code benchmark gain from 68.76 to 78.68 on MTEB Code and says the model can run on-device with about 567MB of active RAM for the full multimodal version on a Google Pixel 11 Pro.

Why it matters: The release shows how a 740M-parameter embedding model can cover text, code, images, audio, and video on local hardware, with memory and storage figures to compare against other on-device options.

AIPhilipp Schmid shares a Hugging Face Space that runs Gemma embedding models in the browser using WebGPU. The demo, a webml-community project, lets users generate embeddings locally without server-side inference.

AIA Hugging Face Space lets users try EmbeddingGemma 2 directly in their browser using WebGPU. The post shares only a demo link, with no further details on model size, benchmarks, or pricing.

AIvLLM announced day-0 support for EmbeddingGemma 2 from Google DeepMind, a bidirectional omni-modal embedding model that maps text, image, audio, video, and interleaved inputs into one vector space. Users can try it with the latest vLLM nightly build using the command vllm serve google/embeddinggemma-2 --runner pooling. The quoted Google post says the model is built on the Gemma 4 architecture and released under Apache 2.0.

AIGoogle's EmbeddingGemma 2 is an open embeddings model for on-device use that covers code, image, video, audio, and text. It comes in modular sizes from 270M text/code to 740M full multimodal, supports Matryoshka truncation down to 128 dimensions, and reports a 14% gain on MTEB Code over v1 under an Apache 2.0 license. The author's post highlights the release and a Hugging Face demo, while the benchmark table compares it with several models.

AIGoogle released EmbeddingGemma 2, a 740M-parameter open embedding model under Apache 2.0 that combines a 270M text model with vision (170M) and audio (300M) encoders. The 270M text model can run locally with 0.5GB of RAM, and the full multimodal model with 1GB, and Unsloth provides GGUF files and fine-tuning support.

AIGoogle's EmbeddingGemma 2 is a 740M-parameter embedding model that outperforms some models more than twice its size while using about 191MB to 567MB of active RAM. It offers an 8K context window, 4x larger than the first generation, and can process up to 5.5 minutes of audio, 29 images, or 58 video frames in one pass.

AIGoogle says EmbeddingGemma 2, paired with Gemma 4, enables efficient on-device retrieval-augmented generation with a lower memory footprint. In this setup, EmbeddingGemma 2 retrieves local files and Gemma 4 reasons over them to produce grounded answers while keeping data private.

AIGoogle has released EmbeddingGemma 2, a compact 740M-parameter model built for on-device and edge applications. It natively embeds text, images, audio, and video into one shared space, enabling cross-format search without manual organization, translation, or labeling.

AIGoogle has released the weights for EmbeddingGemma 2 on Hugging Face and Kaggle. The release includes out-of-the-box support for LiteRT, MediaPipe, LangChain, and LlamaIndex.

AIGoogle's new multimodal embedding setup processes image, audio, and video entirely offline with zero server calls. Its modular design lets developers drop unused vision and audio components to save memory, and flexible dimension sizes cut local database storage by up to 6x. It can also pair with Gemma 4 to build RAG pipelines with minimal memory and processing requirements.

AIGoogle DeepMind released EmbeddingGemma 2, a 740M-parameter embedding model, under an Apache 2.0 license. The post says it is competitive across benchmarks and outperforms some specialist models more than twice its size, and that developers can use it for multimodal search or pair it with Gemma 4 for on-device RAG. Weights are available on Hugging Face and Kaggle.

AIGoogle DeepMind introduced EmbeddingGemma 2, its first natively multimodal open model for on-device embeddings. The model expands beyond text to unify code, images, audio, and video in a shared embedding space.

AIGoogle introduces EmbeddingGemma 2, its first open, natively multimodal embedding model, covering text, code, image, video, and audio tasks. It has a 740M parameter form factor, is positioned for offline, privacy-first RAG when paired with Gemma 4, and the post claims it outperforms some specialist models more than twice its size. Weights are available now on Hugging Face.

AIGoogle DeepMind has released EmbeddingGemma 2, a 740-million-parameter embedding model that maps text, images, audio, and video into a shared space and runs on local hardware under an Apache 2.0 license. Matryoshka Representation Learning lets developers truncate output vectors from 768 dimensions to 512, 256, or 128, and the model supports an 8K-token context window. The model weights are available on Hugging Face and Kaggle, with Gemini Enterprise Agent Platform availability coming soon.

Why it matters: The release shows how a 740M-parameter multimodal embedder runs locally with a 768-to-128 dimension truncation option, useful for judging on-device retrieval designs.

AIDeveloper Niko1221 has open-sourced Strata, an engine that runs a quantized 125B-parameter Qwen3.8-Flash-Next model on consumer GPUs with at least 12GB of VRAM. Strata loads the MoE model into RAM and keeps only frequently used experts in VRAM, and uses a lightweight model for speculative decoding. On an NVIDIA RTX 5070 with 12GB VRAM, the Q2_0 quantization reaches 94 tokens per second for output.

AIHarrison Chase, founder of LangChain, endorsed a post on harnesses with the brief comment "Good take on harnesses." The post, from @zeeg, argues that general coding harnesses like Codex will be superseded by specialized ones and that local models will handle most daily tasks within five years.

AIGoogle DeepMind launched EmbeddingGemma 2, an open-weight 740M parameter model that maps text, images, video frames, and audio into one vector space. The model can run on-device, with about 567MB active RAM for the full multimodal model on a Google Pixel 11 Pro, and is available through Google AI Edge Gallery, Google AI Edge Foresight on Mac, and MediaPipe Tasks, with ML Kit support coming in the weeks ahead.

Why it matters: The post names concrete on-device apps, memory footprints, and latency figures, showing how a multimodal embedding model can power local search without cloud calls.

AILiquidAI has released d1-omni-600M on Hugging Face, a 587M-parameter model that answers named yes/no, choice and score questions over text, images or up to 30 seconds of speech in a single forward pass. It returns typed answers with zero output tokens by reading the model's distribution over options, and is built on LFM2.5-Encoder-350M with a 16,384-token context length. The model is not a chat model and does not generate text.

AIMerve Noyan, a Hugging Face employee, says anyone not using llama.app to run models locally is someone she has nothing to say to. The post is a brief promotional endorsement and includes no features, specifications, or benchmarks.

AITeknium announced that an ESP32 is now running at home, with no further technical details given in the post. The post is a brief update that quotes an @adolandev post about Hermes Gadget, an open SDK for a small device that speaks to a user's own Hermes model and can be tested with a desktop simulator.

AIThe author reports that a dual RTX 5070 Ti setup running Qwen Flash rose from 200 prefill and 10 decode to 2200 prefill and 67 decode, now on a single card, using Strata and a custom PR. The post argues that such consumer-hardware speeds, once limited to top-end machines, could pressure the economics of selling model compute via API.

AIPerplexity has released several open source projects, including the pplx-decider-v1-27b multimodal decision model, the pplx-embed-v2-context-9b-preview contextual embeddings model, and the Lily local inference engine for Apple silicon. The post also lists the 0.6B on-device PII-Tracer classifier with its PII-TRACE benchmark, the WANDR research agent benchmark, and the Numbat and Bumblebee security tools, and says more open source releases are coming soon.

AIThomas Wolf says most personal AI should run on users' devices and praises Sigil's Underdog, which is powered by the Hugging Face Hub. Underdog's announcement says it aims to deliver free, private AI that runs on consumer hardware, with backing from a16z, Khosla Ventures, and other investors.

AIllama.cpp can now run decision models on-device, according to Clément Delangue of Hugging Face. He says the setup is free, fast, and private, and gives the command llama serve -hf ggml-org/Kev-4B-GGUF to start it.

AIGeorgi Gerganov announced that decision models are now available in the latest llama.cpp builds through a new /v1/systemone endpoint. The endpoint supports local, private inference with multiple open models, and more are planned.

AINVIDIA's DGX Spark 64GB configuration will be available from Acer, ASUS, Dell, Gigabyte, HP and MSI on Oct. 23, starting at $4,999. It supports models up to 100 billion parameters on device, and two units can be clustered via NVIDIA Sync Cluster Assistant to pool 128GB of memory and support up to 200 billion parameters. NVIDIA says the clustered setup delivers up to 1.7x the performance of a single system in its Qwen 3.8 27B test.

AINVIDIA announced a new approach to building edge AI through agentic development on NVIDIA Jetson, presented in a broadcast. The post provides no further technical details, figures, or availability information.

AIOllama now supports decision models such as Nimble locally, usable for tasks like ticket triaging, model routing, and content moderation. The model can be installed with ollama pull nimble and accessed through the new local /v1/systemone API, as shown in a real-time Ollama racer demo.

AIA user ran qwen3-vl:32b-instruct locally through Ollama on a 128GB computer to auto-tag 40,000 images in their Eagle app. The image library was collected over 10 years and is intended as a personal asset base for Claude Code-made knowledge videos. The user says the use case makes the 128GB memory purchase feel worthwhile.

AIArtificial Analysis has open-sourced AA-AgentPerf-Local, a tool that replays recorded agent trajectories to measure inference speed on laptops and workstations. Initial results cover NVIDIA DGX Spark, NVIDIA GeForce RTX 5090, AMD Ryzen AI Halo, and MacBook Pro M5 Pro, with the RTX 5090 fastest for models that fit its 32 GB. The source states the tool and leaderboard will expand to more hardware, frameworks, and models.

Why it matters: The source gives per-system completion times and memory bandwidth figures, letting readers compare local hardware for running agentic workloads.

AIUnsloth Desktop can now serve local Laya decision models through a Jev-compatible API, shown in a real-time packing demo where suitcase items update as the user types. The demo runs through Unsloth's Decision API, and the team says more optimizations are coming to speed up local hardware performance.

AIGoogle Cloud argues startups should combine open-weight models with frontier APIs rather than routing every request to one frontier model. It cites Gemma 4, which spans five sizes including a 31B dense model and a 26B A4B Mixture-of-Experts model, released under Apache 2.0. The article's examples report a 44% latency drop for Cue, from 876 ms to 488 ms, and a $0 server cost for BetterSpeak's on-device Gemma 4 E2B.

AIA community-built 4.0 bpw EXL3 quantization of MiniCPM5-2B reduces the quantized model weights to 1.61 GB for local inference. The author reports roughly 68–70 tokens/s on an NVIDIA Tesla T4, and the model runs with ExLlamaV3 and TabbyAPI.

AILiquid AI's post-training team, including Maxime Labonne, Edoardo Mosca, and Jiahui Wang, discusses what makes an on-device agentic model useful. The post says post-training shapes how models learn to use tools, follow instructions, handle longer contexts, and recover when tasks become complex.

AIGoogle says Gemma 4 can now run locally on-device within the Antigravity SDK. Developers can build fully local or hybrid multi-agent workflows that pair cloud models with Gemma 4 agents for auditing, patching, and testing code. The post emphasizes total data privacy and zero API fees, powered by LiteRT.

AILiquid AI announced its on-device Liquid Context layer is now optimized for Snapdragon processors using the Qualcomm Hexagon NPU, letting edge agents learn user routines and share context across devices. Separately, Liquid AI released LFM2-1.2B-Longevity and LFM2-2.6B-Longevity, which the company says often match or outperform much larger frontier LLMs on longevity prediction tasks.

AIDeveloper @Scorp1o_117 used FIT-GGUF to build four MiniCPM5-2B GGUF variants, ranging from about 1.14 GiB to 1.46 GiB, tuned to target file sizes or fidelity tiers. Instead of fixed presets, FIT-GGUF allocates precision tensor by tensor, with Quality, Balanced, Compact, and Mini options, and its generated files matched predicted sizes. Builds are evaluated with KL Divergence and Same-top metrics and are available on Hugging Face.

AITencent Hy Translation has launched, powered by Hy-MT2, supporting 33 languages and 5 Chinese minority languages and dialects. It offers voice and photo translation with full offline, on-device operation requiring no network, and is already live in 12 countries and regions.