Engram offloading to DRAM beats SSD for DeepSeek-V4.1-Flash serving on B200
Original titleEngrams Embedding Entendre: Codesign for Efficient DRAM/SSD Offloading
AISummary
SemiAnalysis tested offloading DeepSeek-V4.1-Flash's Engram embedding table from HBM to host DRAM and to local SSD. On B200 configurations, DRAM delivered more total tokens per dollar and higher P90 interactivity than SSD at every measured point. The report concludes SSD offloading is likely not worth the tradeoff for production serving in its unoptimized setup.
Source: SemiAnalysis · newsletter.semianalysis.comPublished · added here