Evolving Programmatic Skill Networks

The paper introduces the Programmatic Skill Network (PSN), a framework for continual skill acquisition in open-ended embodied environments.

In PSN, skills are executable symbolic programs (e.g., JavaScript for Minecraft) that form a compositional network evolving through experience.

PSN implements three core mechanisms via LLMs: REFLECT for trace-based fault localization over skill compositions, maturity-aware update gating that stabilizes reliable skills while maintaining plasticity for uncertain ones, and canonical structural refactoring with rollback validation to maintain network compactness.

The authors draw structural parallels between PSN's learning dynamics and neural network training, likening fault localization to backpropagation, maturity gating to learning rate scheduling, and refactoring to architecture search.

Experiments on MineDojo and Crafter show that PSN achieves faster and more stable tech tree progression in Minecraft (reaching diamond tools in all runs) and higher cumulative reward in Crafter compared to baselines like Voyager, ADAM, and ReAct.

Ablations demonstrate the importance of the optimizer, maturity gating, and online refactoring.

Cross-model experiments with a weaker open-weight model (Qwen3-Coder-Next) show that PSN's architecture absorbs model noise, with the weaker model requiring deeper credit assignment chains and achieving lower compositional reuse.