The Year in AI Papers
The year/Independent research

Paper 2602.05843

OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions

Hang Yan, Fangzhi Xu, Qiushi Sun, Jinyang Wu, Zixian Huang, Muye Huang, Jingyang Gong, Zichen Ding, Kanzhi Cheng, Yian Wang, Xinyu Che, Zeyi Sun, et al.

Published
Feb 2026
Research lab
Independent
Citations
6
GitHub
33 stars

01 In brief

Summary

ODYSSEYARENA is a benchmark suite for evaluating large language models (LLMs) on long-horizon, active, and inductive interactions, where agents must infer latent transition rules from experience rather than follow explicit instructions.

The paper formalizes four structural primitives—discrete symbolic rules, continuous stochastic dynamics, periodic temporal patterns, and relational graph structures—instantiated in four environments: Turn On Lights, AI Trading, Energy Dispatch, and Repo System.

Two benchmark protocols are provided: ODYSSEYARENA-LITE (120 tasks, step limits of 200 for Turn On Lights and 120 for others) and ODYSSEYARENA-CHALLENGE (tasks exceeding 1,000 steps).

Evaluations of 15+ LLMs show that even the best model, Gemini 3 Pro Preview, achieves only 44.17% average success on Turn On Lights, far below human performance (81.67%).

Key findings include: models excel at deductive reasoning when rules are provided but fail at inductive discovery; performance saturates with extended interaction steps; and failure modes are dominated by exploration limitations (80.85%) and memory constraints (57.45%).

The results highlight a fundamental inductive bottleneck in current LLMs, suggesting that scaling alone is insufficient for autonomous discovery in complex environments.

02 From the paper

Abstract

The rapid advancement of Large Language Models (LLMs) has catalyzed the development of autonomous agents capable of navigating complex environments. However, existing evaluations primarily adopt a deductive paradigm, where agents execute tasks based on explicitly provided rules and static goals, often within limited planning horizons. Crucially, this neglects the inductive necessity for agents to discover latent transition laws from experience autonomously, which is the cornerstone for enabling agentic foresight and sustaining strategic coherence. To bridge this gap, we introduce OdysseyArena, which re-centers agent evaluation on long-horizon, active, and inductive interactions. We formalize and instantiate four primitives, translating abstract transition dynamics into concrete interactive environments. Building upon this, we establish OdysseyArena-Lite for standardized benchmarking, providing a set of 120 tasks to measure an agent's inductive efficiency and long-horizon discovery. Pushing further, we introduce OdysseyArena-Challenge to stress-test agent stability across extreme interaction horizons (e.g., > 200 steps). Extensive experiments on 15+ leading LLMs reveal that even frontier models exhibit a deficiency in inductive scenarios, identifying a critical bottleneck in the pursuit of autonomous discovery in complex environments. Our code and data are available at https://github.com/xufangzhi/Odyssey-Arena