Skip to content
Read the original: PaddlePaddle· Published 36/100AI score36/100

PaddleFormers 1.2 adds DeepSeek-V4 training with 128K+ context support

Original titlePaddleFormers 1.2 is officially out — DeepSeek-V4 training now supported!

AISummary

PaddleFormers 1.2 is released with support for training DeepSeek-V4 and 128K+ long-context training.

The update adds Context Parallel, Packing, Document Mask Attention, and the Muon optimizer, plus ultra-fused mHC, CSA, and HCA operators, DeepEP/HybridEP communication, and lossless FP8 training with AutoSubbatch memory balancing.

The project is presented as fully open-source and is available on GitHub.

Read the original x.com

Source: PaddlePaddle · x.comPublished · added here