Mamba-3 linear model released, outperforming Mamba-2 and Gated DeltaNet
Original titleThe frontier has increasingly shifted to hybrid models - from Qwen to Kimi-Linear and now with NVIDIA's Nemotron-3 Super - that rely on a...
AISummary
Tri Dao announced Mamba-3, which he described as the most powerful linear sequence model to date, as hybrid architectures increasingly rely on strong linear models. The post cites Qwen, Kimi-Linear, and NVIDIA's Nemotron-3 Super as examples of this trend.
According to co-author Albert Gu, Mamba-3 shows noticeable performance gains over Mamba-2 and Gated DeltaNet at all sizes while maintaining speed.
Source: Tri Dao · x.comPublished · added here