Ai2 replaces priority GPU scheduler with time-budget fair-share allocation
Article timeline
The articles in this story. Times are ET.
Ai2@allen_aiAi2 publishes design decisions and rollout lessons for its schedulerAIAi2 has published a blog post on the design decisions behind its scheduler, along with rollout lessons and next priorities. The post links to The text provided does not include further details of the scheduler's features or results.
- Ai2@allen_aiAi2 describes a fair-share GPU scheduler for research budgets
AIAi2 says managers now assign GPU-time budgets to research programs and projects. Its fair-share scheduler compares recent usage with those budgets and moves work from underused allocations ahead in the queue.
- Ai2@allen_aiAi2's new scheduler delivers 98% of owed GPU hours in 30-day test
AIAi2 reports that over a 30-day test of its new scheduler, teams received 98% of the GPU hours they were owed, based on actual demand. Cluster occupancy stayed at 98%, and spare capacity went to interruptible work without drawing down team budgets.
- Ai2@allen_aiAi2 shares engineering behind GPU scheduler that cut queue waits
AIAi2 published the engineering details of its new GPU scheduler, which allocates compute across research teams. On its largest H100 cluster, median queue wait fell from 5 minutes to 24 seconds.
- Ai2@allen_aiAi2 replaces its GPU scheduler after idle jobs hoarded capacity
AIAi2 says its old scheduler made every scheduled workload eventually run at HIGH priority. Researchers kept idle jobs running to reserve GPUs for experiments, because the incentives rewarded holding capacity even with no active work.
- Hugging Face BlogAi2 replaces priority scheduler with GPU time budgets for cluster allocation
AIAi2's AI Infrastructure team replaced its priority-based GPU cluster scheduler with a system using GPU time budgets, hierarchical fair-share allocation, and a time-slicing contract. The team says the change turns decisions about how much GPU time each research project receives into a transparent administrative budgeting process. Its clusters, which range from 88 to 1024 GPUs including H100, B200, and B300 units, serve about 150 researchers facing demand two to three times available capacity.
- Ai2 (Allen Institute for AI)Ai2 describes GPU time budgets that replaced its priority-based cluster scheduler
AIAi2's AI Infrastructure team replaced its priority-based scheduler for GPU clusters with GPU time budgets, hierarchical fair-share allocation, and a time-slicing contract. The team says the change moved debates over how much GPU time each research project deserves from case-by-case operational decisions into a transparent budgeting process. The clusters range from 88 to 1024 GPUs across NVIDIA H100, B200, and B300 hardware, and serve about 150 internal researchers.