Infer-forge: Three-layer agent system for SGLang inference optimization
Original title🚀 Infer-forge: Harness, Loop, and Graph Engineering Around SGLang
AISummary
Ant OSS built Infer-forge, a three-layer system of Harness, Task Loop, and Task Graph that runs long SGLang inference optimization work through agents while keeping provenance.
Peak Tasks in flight rose from 2 to 9, and median Task lifetime grew from 10 hours to 28 hours. The agent independently ran a full serving project on DeepSeek-V4-Pro, splitting the work into 38 verified pieces and catching kernel silent corruption on its own.
Source: LMSYS Org · x.com