Paper 2601.20245
How AI Impacts Skill Formation
- Published
- Jan 2026
- Research lab
- Anthropic
- Citations
- 24
- GitHub
- Not linked
01 In brief
Summary
This study examines how AI assistance affects skill formation in software engineering.
In a randomized experiment, 52 developers learned a new asynchronous Python library (Trio) with or without an AI assistant.
Results show that AI use significantly reduced quiz scores measuring conceptual understanding, code reading, and debugging (17% lower, Cohen's d=0.738, p=0.010), without significant average time savings.
Qualitative analysis of screen recordings identified six AI interaction patterns; three high-scoring patterns (Conceptual Inquiry, Hybrid Code-Explanation, Generation-Then-Comprehension) involved cognitive engagement and preserved learning, while low-scoring patterns (AI Delegation, Progressive AI Reliance, Iterative AI Debugging) relied heavily on AI and scored below 40%.
The control group encountered more errors, which correlated with better skill development.
The authors conclude that AI-enhanced productivity is not a shortcut to competence and recommend careful AI adoption to preserve skill formation, especially in safety-critical domains.
02 From the paper
Abstract
AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.