The Year in AI Papers
The year/Independent research

Paper 2605.13779

MinT: Managed Infrastructure for Training and Serving Millions of LLMs

Mind Lab, :, Song Cao, Vic Cao, Andrew Chen, Kaijie Chen, Cleon Cheng, Steven Chiang, Kaixuan Fan, Hera Feng, Huan Feng, Arthur Fu, et al.

Published
May 2026
Research lab
Independent
Citations
1
GitHub
Not linked

01 In brief

Summary

MinT is a managed infrastructure system for Low-Rank Adaptation (LoRA) post-training and online serving, designed for settings where many trained policies are produced over a small number of expensive base-model deployments.

Instead of materializing full checkpoints, MinT keeps the base model resident and moves exported LoRA adapter revisions through rollout, update, export, evaluation, serving, and rollback.

This adapter-revision path hides distributed training, serving, scheduling, and data movement behind a service interface.

MinT scales along three axes: Scale Up supports LoRA RL on frontier-scale dense and MoE architectures, validated beyond 1T total parameters; Scale Down minimizes handoff by moving only the adapter (less than 1% of base size in rank-1 settings), reducing handoff step by 18.3× on a 4B dense model and 2.85× on a 30B MoE model, and shortening concurrent multi-policy GRPO wall time by 1.77× and 1.45× respectively; Scale Out expands the policy namespace to a million-scale adapter catalog, validated with 1,000,000 packed adapters, while keeping engine-local execution bounded.

Packed MoE LoRA tensors improve live engine loading by 8.5–8.7×, and admission-aware readiness gates expose new adapters only after prewarm.

MinT provides a Tinker-compatible API and public cookbook recipes for reproducibility.

02 From the paper

Abstract

We present MindLab Toolkit (MinT), a managed infrastructure system for Low-Rank Adaptation (LoRA) post-training and online serving. MinT targets a setting where many trained policies are produced over a small number of expensive base-model deployments. Instead of materializing each policy as a merged full checkpoint, MinT keeps the base model resident and moves exported LoRA adapter revisions through rollout, update, export, evaluation, serving, and rollback, hiding distributed training, serving, scheduling, and data movement behind a service interface. MinT scales this path along three axes. Scale Up extends LoRA RL to frontier-scale dense and MoE architectures, including MLA and DSA attention paths, with training and serving validated beyond 1T total parameters. Scale Down moves only the exported LoRA adapter, which can be under 1% of base-model size in rank-1 settings; adapter-only handoff reduces the measured step by 18.3x on a 4B dense model and 2.85x on a 30B MoE, while concurrent multi-policy GRPO shortens wall time by 1.77x and 1.45x without raising peak memory. Scale Out separates durable policy addressability from CPU/GPU working sets: a tensor-parallel deployment supports 10^6-scale addressable catalogs (measured single-engine sweeps through 100K) and thousand-adapter active waves at cluster scale, with cold loading treated as scheduled service work and packed MoE LoRA tensors improving live engine loading by 8.5-8.7x. MinT thus manages million-scale LoRA policy catalogs while training and serving selected adapter revisions over shared 1T-class base models.