PaddleFormers 1.2 adds DeepSeek-V4 training with 128K+ context support
Original titlePaddleFormers 1.2 is officially out — DeepSeek-V4 training now supported!
AISummary
PaddleFormers 1.2 is released with support for training DeepSeek-V4 and 128K+ long-context training.
The update adds Context Parallel, Packing, Document Mask Attention, and the Muon optimizer, plus ultra-fused mHC, CSA, and HCA operators, DeepEP/HybridEP communication, and lossless FP8 training with AutoSubbatch memory balancing.
The project is presented as fully open-source and is available on GitHub.
Source: PaddlePaddle · x.comPublished · added here