Skip to content
Read the original: LMSYS Org· lmsysorg·Published AI score34/100

Infer-forge: Three-layer agent system for SGLang inference optimization

Original title🚀 Infer-forge: Harness, Loop, and Graph Engineering Around SGLang

AISummary

Ant OSS built Infer-forge, a three-layer system of Harness, Task Loop, and Task Graph that runs long SGLang inference optimization work through agents while keeping provenance.

Peak Tasks in flight rose from 2 to 9, and median Task lifetime grew from 10 hours to 28 hours. The agent independently ran a full serving project on DeepSeek-V4-Pro, splitting the work into 38 verified pieces and catching kernel silent corruption on its own.

Read the original x.com

Source: LMSYS Org · x.com