A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers

This survey reframes the development of Scientific Large Language Models (Sci-LLMs) as a co-evolution between models and their data substrate, providing a data-centric synthesis across six scientific domains (physics, chemistry, materials science, life sciences, astronomy, and Earth science).

It introduces a unified taxonomy of scientific data and a hierarchical model of scientific knowledge, highlighting the multimodal, cross-scale, and domain-specific challenges that differentiate scientific corpora from general NLP datasets.

The survey systematically reviews recent Sci-LLMs, from general-purpose foundations to specialized models, and analyzes over 270 pre-/post-training datasets and over 190 evaluation benchmarks.

It identifies persistent issues in scientific data development, such as scarcity of experimental data, over-reliance on text, and multi-level biases, and discusses emerging solutions like semi-automated annotation pipelines and expert validation.

The paper outlines a paradigm shift toward closed-loop systems where autonomous agents based on Sci-LLMs actively experiment, validate, and contribute to a living knowledge base, providing a roadmap for building trustworthy, continually evolving AI systems for scientific discovery.

The evolution of Sci-LLMs is traced through four phases: transfer learning, scaling, instruction-following, and agentic science, with a noted trend toward smaller parameter sizes (7B-13B) and the dominance of open-source base models like LLaMA and Qwen.

Evaluation is shifting from static exams to process- and discovery-oriented assessments, with advanced protocols like LLM-as-a-Judge and test-time learning.

The survey concludes by outlining future directions including integrated data ecosystems, automated data standardization, comprehensive evaluation systems, and autonomous scientific agents, emphasizing the need for ethical governance and responsible AI innovation.

The work is a…