Hermes now has a local backend!
There's support for Unsloth UD-Q4_K_XL and UD-Q4_K_M quants for DeepSeek-V4-Flash, Qwen3.8-27B, 3.6-35B-A3B and Qwen3.8-Flash-Next in Hermes!
Hermes adds local backend supporting Unsloth GGUF quantized models
AISummary
Hermes now includes a local backend that runs Unsloth UD-Q4_K_XL and UD-Q4_K_M quants for DeepSeek-V4-Flash, Qwen3.8-27B, Qwen3.6-35B-A3B, and Qwen3.8-Flash-Next. The post says these models can be run locally in one click, with Unsloth GGUFs supported.
Post on XView on X
@danielhanchen
You can now run Unsloth GGUFs locally in one-click via Hermes! ✨ Qwen3.8-27B, Qwen3.8-Flash, DeepSeek-V4-Flash and more are all supported.
Source: Daniel Han · x.comPublished · added here