Paper 2603.03790
T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 24 stars
01 In brief
Summary
The paper introduces Structure of Thought (SoT), a prompting technique that guides LLMs to construct intermediate text structures (nodes and links) before answering, consistently improving performance across eight text-processing tasks and three model families.
Building on this, the authors present T2S-Bench, the first benchmark for evaluating text-to-structure capabilities, containing 1.8K samples across six scientific domains and 32 structural types, with a training set (T2S-Train-1.2k), a multi-hop reasoning test set (T2S-Bench-MR, 500 samples), and an end-to-end extraction set (T2S-Bench-E2E, 87 samples).
Evaluation of 45 models shows average exact match of 52.1% on T2S-Bench-MR, with Gemini-2.5-Pro leading at 81.4% EM; node extraction remains challenging, with the best model achieving only 58.1% accuracy.
Fine-tuning Qwen2.5-7B-Instruct on T2S-Train-1.2k yields an average +8.6% improvement across downstream tasks, and SoT alone provides +5.7% gains.
The benchmark reveals that structure extraction, especially node identification, is a major bottleneck, and that structuring skills correlate with and transfer to general long-context reasoning performance.
The dataset and evaluation code are publicly released.
The work highlights explicit text structuring as a universal intermediate representation for improving LLM text processing, with potential applications in document-centric workflows and implications for auditability and dual-use risks.
02 From the paper
Abstract
Think about how human handles complex reading tasks: marking key points, inferring their relationships, and structuring information to guide understanding and responses. Likewise, can a large language model benefit from text structure to enhance text-processing performance? To explore it, in this work, we first introduce Structure of Thought (SoT), a prompting technique that explicitly guides models to construct intermediate text structures, consistently boosting performance across eight tasks and three model families. Building upon this insight, we present T2S-Bench, the first benchmark designed to evaluate and improve text-to-structure capabilities of models. T2S-Bench includes 1.8K samples across 6 scientific domains and 32 structural types, rigorously constructed to ensure accuracy, fairness, and quality. Evaluation on 45 mainstream models reveals substantial improvement potential: the average accuracy on the multi-hop reasoning task is only 52.1%, and even the most advanced model achieves 58.1% node accuracy in end-to-end extraction. Furthermore, on Qwen2.5-7B-Instruct, SoT alone yields an average +5.7% improvement across eight diverse text-processing tasks, and fine-tuning on T2S-Bench further increases this gain to +8.6%. These results highlight the value of explicit text structuring and the complementary contributions of SoT and T2S-Bench. Dataset and eval code have been released at https://t2s-bench.github.io/T2S-Bench-Page/.