Monthly collection
The papers that drew the most attention during this month of AI research.
arXiv.org
The paper introduces the Very Big Video Reasoning (VBVR) suite, comprising a large-scale dataset and benchmark for video reasoning. VBVR-Dataset contains 2,015,000 images and 1,007,500 video clips across 200 tasks, organized by five cognitive faculties (Abstraction, Knowledge, Perception, Spatiality, Transformation), and is approximately 1000x larger than…
Maijunxian Wang, Ruisi Wang, Juyi Lin, Ran Ji, et al.Qwen
OPUS is a dynamic data selection framework for LLM pre-training that scores candidate samples by projecting their optimizer-induced effective updates onto a target direction from a stable, in-distribution proxy. It addresses the limitations of static filters and raw-gradient-based dynamic methods by aligning selection with the actual update geometry of…
Shaobo Wang, Xuan Ouyang, Tianyi Xu, Yuzheng Hu, et al.arXiv.org
The paper introduces Weak-Driven Learning, a post-training paradigm that repurposes weak historical checkpoints as corrective signals to strengthen a strong language model, contrasting with knowledge distillation that imitates stronger teachers. The authors instantiate this with WMSS (Weak agents Make Strong agents Stronger), which combines Weak-Driven…
Zehao Chen, Gongxun Li, Tianxiang Ai, Zixuan Huang, et al.Moonshot AI
Kimi K2.5 is an open-source multimodal agentic model that jointly optimizes text and vision through techniques including joint pre-training, zero-vision SFT, and joint reinforcement learning. It introduces Agent Swarm, a parallel agent orchestration framework using Parallel-Agent Reinforcement Learning (PARL) to decompose tasks into heterogeneous…
Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, et al.arXiv.org
ERNIE 5.0 is a natively autoregressive foundation model from Baidu for unified multimodal understanding and generation across text, image, video, and audio. All modalities are trained from scratch under a unified next-group-of-tokens prediction objective, using an ultra-sparse mixture-of-experts (MoE) architecture with modality-agnostic expert routing. A…
Haifeng Wang, Hua Wu, Tian Wu, Yu Sun, et al.arXiv.org
The paper investigates why large reasoning models (LRMs) produce lengthy, redundant chains of thought (CoTs) despite shorter chains often being more accurate. The authors discover that LRMs implicitly know when to stop thinking, but this capability is obscured by current sampling paradigms like pass@1. They introduce SAGE (Self-Aware Guided Efficient…
Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, et al.Research paper
The paper introduces Feature Activation Coverage (FAC), a metric that measures data diversity in an interpretable feature space derived from Sparse Autoencoders (SAEs) trained on LLM internal activations. The authors propose FAC Synthesis, a framework that identifies task-relevant features missing from a seed dataset and generates synthetic samples to…
Zhongzhi Li, Xuansheng Wu, Yijiang Li, Lijie Hu, et al.Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval
SQuTR is a benchmark for evaluating spoken query to text retrieval under controlled acoustic noise. It aggregates 37,317 unique queries from six English and Chinese text retrieval datasets (FiQA, HotpotQA, Natural Questions, MedicalRetrieval, DuRetrieval, T2Retrieval), synthesizing speech with CosyVoice-3 using 200 speakers. Four acoustic conditions…
Yuejie Li, Ke Yang, Yueying Hua, Berlin Chen, et al.arXiv.org
VESPO addresses high variance in off-policy reinforcement learning for large language models by deriving a principled importance weight reshaping kernel from a variational formulation. It recasts weight reshaping as a measure change to an implicit proposal distribution, solving a KL-regularized objective with a variance constraint to obtain a closed-form…
Guobin Shen, Chenxiao Zhao, Xiang Cheng, Lei Huang, et al.arXiv.org
TermiGen is an end-to-end pipeline for synthesizing verifiable terminal environments and robust expert trajectories to train open-weight LLMs for terminal tasks. It addresses two key challenges: scarcity of high-fidelity executable environments and exposure bias from standard expert trajectories. Phase I uses a multi-agent system to generate diverse tasks…
Kaijie Zhu, Yuzhou Nie, Yijiang Li, Yiming Huang, et al.Z.ai / GLM
GLM-5, developed by Zhipu AI and Tsinghua University, is a next-generation foundation model that shifts from vibe coding to agentic engineering. It builds on the ARC (agentic, reasoning, coding) capabilities of its predecessor, GLM-4.7, and introduces DeepSeek Sparse Attention (DSA) to reduce training and inference costs while maintaining long-context…
GLM-5-Team, :, Aohan Zeng, Xin Lv, et al.arXiv.org
This paper proposes that a General World Model must be grounded in the Trinity of Consistency: Modal Consistency (semantic interface), Spatial Consistency (geometric basis), and Temporal Consistency (causal engine). The authors systematically review the evolution of multimodal learning from specialized modules to unified architectures, arguing that…
Jingxuan Wei, Siyuan Li, Yuhang Xu, Zheng Sun, et al.arXiv.org
Code2World is a vision-language coder that predicts the next GUI state by generating renderable HTML code, combining high visual fidelity with structural controllability. To address data scarcity, the authors constructed AndroidCode, a corpus of over 80K screen-action pairs, by translating GUI trajectories from AndroidControl into HTML using GPT-5 and…
Yuhao Zheng, Li'an Zhong, Yi Wang, Rui Dai, et al.Research paper
Step 3.5 Flash is a sparse Mixture-of-Experts (MoE) language model with 196B total parameters and 11B active parameters, designed to deliver frontier-level agentic intelligence with high computational efficiency. Its architecture uses a 3:1 interleaved Sliding Window/Full Attention layout, head-wise gated attention, and Multi-Token Prediction (MTP-3) to…
Ailin Huang, Ang Li, Aobo Kong, Bin Wang, et al.arXiv.org
The paper argues that a multi-agent system built from large language models cannot simultaneously achieve continuous self-evolution, complete isolation from external feedback, and safety invariance—a combination termed the self-evolution trilemma. Using an information-theoretic framework, safety is formalized as the KL divergence from an anthropic value…
Chenxu Wang, Chaozhuo Li, Songyang Liu, Zejian Chen, et al.arXiv.org
QuantaAlpha is an evolutionary alpha mining framework that treats each end-to-end mining run as a trajectory and improves factors via trajectory-level mutation and crossover. It addresses limitations in existing agentic systems, such as fragile controllability, limited trustworthiness, and constrained exploration, by localizing suboptimal steps for…
Jun Han, Shuo Zhang, Wei Li, Yifan Dong, et al.arXiv.org
MOVA is an open-source model for generating synchronized video and audio, including lip-synced speech, sound effects, and music. It uses a Mixture-of-Experts architecture with 32B total parameters (18B active) and supports Image-Text to Video-Audio generation. The model couples a pretrained A14B video DiT and a 1.3B audio DiT via a 2.6B bidirectional…
SII-OpenMOSS Team, :, Donghua Yu, Mingshu Chen, et al.arXiv.org
UI-Venus-1.5 is a unified, end-to-end GUI agent family (2B, 8B, and 30B-A3B variants) built on Qwen3-VL, designed for robust real-world applications. It introduces three key advances over UI-Venus-1.0: a Mid-Training stage using 10B tokens from 30+ datasets to establish foundational GUI semantics; Online Reinforcement Learning with full-trajectory rollouts…
Venus Team, Changlong Gao, Zhangxuan Gu, Yulin Liu, et al.arXiv.org
FASA is a training-free framework for query-aware token eviction in LLMs, addressing KV cache memory bottlenecks. It exploits functional sparsity in RoPE frequency chunks (FCs), where a small subset of 'dominant' FCs shows high contextual agreement with full attention heads. FASA uses a two-stage process: Token Importance Prediction (TIP) selects critical…
Yifei Wang, Yueqi Wang, Zhenrui Yue, Huimin Zeng, et al.Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
The paper introduces dLLM, an open-source framework that standardizes the training, inference, and evaluation of diffusion language models (DLMs). It provides modular trainers for Masked Diffusion (MDLM) and Block Diffusion (BD3LM), a plug-and-play sampler abstraction supporting efficient decoding like Fast-dLLM, and a unified evaluation pipeline that…
Zhanhui Zhou, Lingjie Chen, Hanghang Tong, Dawn SongarXiv.org
The paper introduces Diagnostic-driven Progressive Evolution (DPE), a closed-loop training framework for Large Multimodal Models (LMMs) that addresses limitations of static data and heuristic self-evolution methods. DPE iteratively performs diagnosis, targeted data generation, and reinforcement learning. A diagnostic agent identifies capability blind spots…
Hongrui Jia, Chaoya Jiang, Yongrui Heng, Shikun Zhang, et al.arXiv.org
The paper addresses the Modality Gap in multimodal contrastive learning, where embeddings of different modalities for the same semantics occupy offset regions. Prior methods rely on isotropic assumptions, which are flawed. The authors propose the Fixed-frame Modality Gap Theory, decomposing the gap into stable biases (PMB, POB) and anisotropic residuals,…
Xiaomin Yu, Yi Xin, Yuhui Zhang, Wenjie Zhang, et al.arXiv.org
The paper introduces ADE-CoT, a test-time scaling framework for image editing that improves efficiency and performance. It addresses three issues with applying Image-CoT to editing: inefficient fixed sampling budgets, unreliable early-stage verification using general MLLM scores, and redundant results from large-scale sampling. ADE-CoT uses…
Xiangyan Qu, Zhenlong Yuan, Jing Tang, Rui Chen, et al.arXiv.org
VidVec introduces a method to leverage generative Multimodal Large Language Models (MLLMs) for video–text retrieval without visual fine-tuning. The authors show that intermediate layers of off-the-shelf video MLLMs (e.g., VideoLLaMA3-7B) encode substantial retrieval-relevant information, outperforming final-layer embeddings in zero-shot settings. They…
Issar Tzachor, Dvir Samuel, Rami Ben-AriarXiv.org
The paper introduces VDR-Bench, a benchmark of 2,000 VQA instances designed to evaluate Vision-DeepResearch systems under realistic conditions. It identifies two flaws in existing benchmarks: they are not visual-search-centric (answers can be inferred from text cues or model priors) and rely on idealized retrieval (near-exact whole-image matching).…
Yu Zeng, Wenxuan Huang, Zhen Fang, Shuang Chen, et al.arXiv.org
MobilityBench is a scalable benchmark for evaluating LLM-based route-planning agents in real-world mobility scenarios, built from 100,000 anonymized queries from Amap across 22 countries and over 350 cities. It covers 11 task scenarios in four families: Basic Information Retrieval, Route-Dependent Information Retrieval, Basic Route Planning, and…
Zhiheng Song, Jingshuai Zhang, Chuan Qin, Chao Wang, et al.NVIDIA
This paper addresses the lack of transparency in training data for terminal-capable LLMs by introducing Terminal-Task-Gen, a synthetic task generation pipeline, and Terminal-Corpus, a large-scale open-source dataset. The authors fine-tune Qwen3 models (8B, 14B, 32B) to create Nemotron-Terminal, achieving substantial gains on Terminal-Bench 2.0: 8B improves…
Renjie Pi, Grace Lam, Mohammad Shoeybi, Pooya Jannaty, et al.arXiv.org
WIDESEEK-R1 explores width scaling for broad information seeking using a lead-agent–subagent framework trained via multi-agent reinforcement learning (MARL). The system uses a shared LLM with isolated contexts and specialized tools, where the lead agent decomposes tasks and delegates subtasks to parallel subagents. Training on a curated 20k dataset of…
Zelai Xu, Zhexuan Xu, Ruize Zhang, Chunyang Zhu, et al.arXiv.org
CUDA Agent is a large-scale agentic reinforcement learning (RL) system for generating high-performance CUDA kernels. It addresses the gap between LLMs and compiler-based systems like torch.compile by combining a scalable data synthesis pipeline (crawling seed operators, LLM-based combinatorial fusion, and rubric-based filtering to create…
Weinan Dai, Hanlin Wu, Qiying Yu, Huan-ang Gao, et al.arXiv.org
This paper presents the first systematic study on the effectiveness of Vision Language Models (MLLMs) for code understanding by representing source code as images. The authors evaluate seven MLLMs across four tasks (code summarization, completion, clone detection, and question answering) with compression ratios from 1x to 8x and rendering strategies…
Yuling Shi, Chaoxiang Xie, Zhensu Sun, Yeheng Chen, et al.arXiv.org
Composition-RL addresses the issue of 'solve all' prompts (pass rate 1) that become prevalent during RLVR training, reducing effective data size. The method automatically composes multiple existing verifiable prompts into new, harder ones using Sequential Prompt Composition (SPC), which modifies one prompt with a variable from another's answer and links…
Xin Xu, Clive Bai, Kai Yang, Tianhao Chen, et al.arXiv.org
SkillNet is an open infrastructure for creating, evaluating, and organizing AI skills at scale, addressing the lack of systematic skill consolidation in current AI agents. It structures skills within a three-layer ontology (taxonomy, relation graph, package library) and supports automated skill creation from heterogeneous sources like execution…
Yuan Liang, Ruobin Zhong, Haoming Xu, Chen Jiang, et al.arXiv.org
SWE-rebench V2 introduces a language-agnostic automated pipeline for harvesting executable real-world software engineering (SWE) tasks at scale, addressing the scarcity of large-scale training data for reinforcement learning (RL). The pipeline mines pull request histories, synthesizes repository-specific installation and test procedures via an interactive…
Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander GolubevarXiv.org
AORCHESTRA is an agentic framework that automates sub-agent creation for complex, long-horizon tasks. It introduces a unified four-tuple abstraction (Instruction, Context, Tools, Model) to model any agent, enabling on-demand specialization. A central orchestrator decomposes tasks, curates context, selects tools and models, and delegates execution to…
Jianhao Ruan, Zhihao Xu, Yiran Peng, Fashen Ren, et al.arXiv.org
DFlash is a speculative decoding framework that uses a lightweight block diffusion model for parallel drafting, addressing the sequential bottleneck of autoregressive LLM decoding. It extracts hidden features from the target LLM and injects them into every draft layer's KV cache, conditioning the draft model to predict future token blocks in parallel. This…
Jian Chen, Yesheng Liang, Zhijian LiuOpen MIND
RPG-Encoder generalizes the Repository Planning Graph (RPG) into a unified, high-fidelity representation for repository reasoning, closing the loop between comprehension and generation. It addresses the disconnect caused by fragmented representations (API documentation lacks structure, dependency graphs lack semantics) by encoding raw code into a dual-view…
Jane Luo, Chengyu Yin, Xin Zhang, Qingtao Li, et al.Research paper
InternAgent-1.5 is a unified agentic framework for long-horizon autonomous scientific discovery, developed by the InternScience Team at Shanghai Artificial Intelligence Laboratory. It addresses limitations of existing AI Scientist systems, such as domain-specific designs and limited long-horizon operation, by integrating three coordinated subsystems:…
Shiyang Feng, Runmin Ma, Xiangchao Yan, Yue Fan, et al.Meta AI
AIRS-Bench is a benchmark suite of 20 tasks sourced from 17 state-of-the-art machine learning papers, designed to evaluate LLM agents on the full research lifecycle (idea generation, implementation, experimentation, analysis) without providing baseline code. Tasks span seven categories: language modeling, math, code, molecules/proteins, time series, text…
Alisia Lupidi, Bhavul Gauri, Thomas Simon Foster, Bassel Al Omari, et al.arXiv.org
DeepGen 1.0 is a lightweight 5B-parameter unified multimodal model (3B VLM + 2B DiT) for image generation and editing, achieving performance competitive with or surpassing much larger models. It introduces Stacked Channel Bridging (SCB), which fuses features from six uniformly distributed VLM layers with learnable 'think tokens' to provide the DiT with…
Dianyi Wang, Ruihang Li, Feng Han, Chaofan Ma, et al.arXiv.org
UniReason is a unified framework for text-to-image (T2I) generation and image editing, built on the Bagel architecture. It addresses limitations of existing methods by incorporating world knowledge-enhanced textual reasoning before synthesis and fine-grained editing-like visual refinement after initial generation. The framework uses two complementary…
Dianyi Wang, Chaofan Ma, Feng Han, Size Wu, et al.arXiv.org
The paper studies data efficiency for training Multimodal Process Reward Models (MPRMs), which provide step-level supervision for visual reasoning in Multimodal Large Language Models. Training MPRMs typically requires large-scale Monte Carlo (MC)-annotated corpora, which is computationally expensive. The authors find that random subsampling of training…
Jinyuan Li, Chengsong Huang, Langlin Huang, Shaoyang Xu, et al.arXiv.org
The paper introduces F-GRPO, a method to mitigate distribution sharpening in Reinforcement Learning with Verifiable Rewards (RLVR) using group-relative objectives like GRPO, DAPO, and CISPO. The authors derive a non-monotonic tail-miss probability, showing that intermediate group sizes can maximize active updates that miss rare-correct solutions, while…
Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov, Viacheslav Sinii, et al.arXiv.org
SKILLRL is a framework for LLM agents that improves policy learning by distilling raw interaction trajectories into a hierarchical skill library (SKILLBANK) and co-evolving it with the agent's policy during reinforcement learning. It uses a teacher model to extract general and task-specific skills from both successful and failed episodes, achieving 10-20x…
Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, et al.arXiv.org
The paper introduces Experiential Reinforcement Learning (ERL), a training paradigm for language models that embeds an explicit experience–reflection–consolidation loop into reinforcement learning. In ERL, the model first generates an initial attempt, receives environmental feedback, and then produces a self-reflection that guides a refined second attempt.…
Taiwei Shi, Sihao Chen, Bowen Jiang, Linxin Song, et al.arXiv.org
The paper introduces Chain of Mindset (CoM), a training-free agentic framework for LLM reasoning that enables step-level adaptive switching among four cognitive mindsets: Spatial, Convergent, Divergent, and Algorithmic. A Meta-Agent dynamically selects the optimal mindset based on the evolving reasoning state, while a bidirectional Context Gate filters…
Tianyi Jiang, Arctanx An, Hengyi Feng, Naixin Zhai, et al.Research paper
The paper investigates the latent planning horizon of Large Language Models (LLMs) during Chain-of-Thought (CoT) reasoning. The authors introduce Tele-Lens, a probing method using low-rank adapters to predict teleological information (subsequent tokens, final answers, reasoning length) from hidden states across 12 diverse tasks. Empirical results reveal…
Liyan Xu, Mo Yu, Fandong Meng, Jie ZhouarXiv.org
The paper introduces Recurrent-Depth VLA (RD-VLA), a Vision-Language-Action model that enables adaptive test-time compute via latent iterative reasoning. Unlike token-based Chain-of-Thought methods, RD-VLA uses a weight-tied recurrent action head that refines a latent scratchpad over multiple iterations, keeping memory constant. The architecture includes a…
Yalcin Tur, Jalal Naghiyev, Haoquan Fang, Wei-Chuan Tsai, et al.arXiv.org
LLaDA2.1 introduces a novel decoding scheme for discrete diffusion language models (dLLMs) that combines Mask-to-Token (M2T) and Token-to-Token (T2T) editing, controlled by dual probability thresholds. This allows two operating modes: Speedy Mode (S Mode) lowers the M2T threshold for faster generation, relying on T2T to correct errors, while Quality Mode…
Tiwei Bie, Maosong Cao, Xiang Cao, Bingsen Chen, et al.arXiv.org
The paper introduces SPIDER-SENSE, a framework for defending LLM-based autonomous agents against security threats. It argues that existing mandatory, stage-wise security checks are inefficient and proposes Intrinsic Risk Sensing (IRS), which embeds risk awareness into the agent's execution flow, triggering defenses only when a risk is perceived. Once…
Zhenxiong Yu, Zhi Yang, Zhiheng Jin, Shuhe Wang, et al.arXiv.org
The paper introduces Generalized On-Policy Distillation (G-OPD), a framework that extends standard on-policy distillation (OPD) by adding a flexible reference model and a reward scaling factor λ. The authors show that OPD is a special case of dense KL-constrained RL with equal reward and KL weights. In G-OPD, λ controls the relative weight of the reward…
Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, et al.arXiv.org
MedXIAOHE is a medical vision-language foundation model from ByteDance that achieves state-of-the-art performance across 30+ medical benchmarks, surpassing leading closed-source systems like GPT-5.2 Thinking and Gemini 3.0 Pro. The model uses a Seed-ViT vision encoder and a large language model, trained via a three-stage pipeline. Continual pretraining…
Baorong Shi, Bo Cui, Boyuan Jiang, Deli Yu, et al.Google DeepMind
MARS (Modular Agent with Reflective Search) is a framework for automating AI research, specifically addressing the bottleneck of complex machine learning engineering (MLE) tasks. It uses three pillars: Budget-Aware Planning via cost-constrained Monte Carlo Tree Search (MCTS) to balance performance with execution cost, Modular Construction through a…
Jiefeng Chen, Bhavana Dalvi Mishra, Jaehyun Nam, Rui Meng, et al.arXiv.org
The paper introduces 3DiMo, a framework for 3D-aware human motion control in video generation. It addresses limitations of existing methods that use 2D poses (viewpoint-bound) or explicit 3D models like SMPL (inaccurate, overriding generator priors). 3DiMo jointly trains a Transformer-based motion encoder with a pretrained DiT video generator to distill…
Zhixue Fang, Xu He, Songlin Tang, Haoxian Zhang, et al.arXiv.org
SKILLSBENCH is a benchmark for evaluating Agent Skills, which are structured packages of procedural knowledge that augment LLM agents at inference time. The benchmark contains 87 tasks across 8 domains, each with curated Skills and deterministic verifiers. In an evaluation of 18 model–harness configurations, curated Skills raised the average pass rate from…
Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, et al.Qwen
Qwen3-Coder-Next is an 80-billion-parameter open-weight language model with only 3 billion active parameters per forward pass, designed for coding agents. It is built on Qwen3-Next with hybrid attention and Mixture-of-Experts, and trained via a staged pipeline: continued pretraining on code and agent-centric data, supervised fine-tuning, and reinforcement…
Ruisheng Cao, Mouxiang Chen, Jiawei Chen, Zeyu Cui, et al.arXiv.org
MemSkill reframes LLM agent memory operations as learnable, evolvable memory skills. It maintains a shared skill bank where each skill provides structured guidance (purpose, when to use, how to apply, constraints) for extracting, consolidating, or revising memories. A controller, trained with reinforcement learning (PPO), selects a Top-K set of relevant…
Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, et al.Conference of the European Chapter of the Association for Computational Linguistics
This paper introduces AudioSAE, the first large-scale application of Sparse Autoencoders (SAEs) to audio models, training them on all encoder layers of Whisper and HuBERT. The authors evaluate feature stability, interpretability, and practical utility. Over 50% of features remain consistent across random seeds, and reconstruction quality is preserved. SAE…
Georgii Aparin, Tasnima Sadekova, Alexey Rukhovich, Assel Yermekova, et al.arXiv.org
GigaBrain-0.5M* is a vision-language-action (VLA) model trained via world model-based reinforcement learning, built on GigaBrain-0.5, which is pretrained on over 10,000 hours of robotic manipulation data and ranks first on the RoboChallenge benchmark. The model integrates RAMP (Reinforcement leArning via world Model-conditioned Policy), a four-stage…
GigaBrain Team, Boyuan Wang, Bohan Li, Chaojun Ni, et al.Google DeepMind
Unified Latents (UL) is a framework for learning latent representations regularized by a diffusion prior and decoded by a diffusion model. The encoder outputs a deterministic latent, which is noised to a fixed minimum noise level (log-SNR of 5), linking the encoder's output noise to the prior's precision. The training objective combines a diffusion prior…
Jonathan Heek, Emiel Hoogeboom, Thomas Mensink, Tim SalimansQwen
SWE-Universe is a framework for automatically constructing real-world software engineering (SWE) verifiable environments from GitHub pull requests (PRs) at a million scale. It addresses challenges of low production yield, weak verifiers, and prohibitive cost using a building agent powered by a custom-trained MoE model (Qwen-Next-80B-A3B). The agent uses…
Mouxiang Chen, Lei Zhang, Yunlong Feng, Xuwu Wang, et al.arXiv.org
ODYSSEYARENA is a benchmark suite for evaluating large language models (LLMs) on long-horizon, active, and inductive interactions, where agents must infer latent transition rules from experience rather than follow explicit instructions. The paper formalizes four structural primitives—discrete symbolic rules, continuous stochastic dynamics, periodic…
Hang Yan, Fangzhi Xu, Qiushi Sun, Jinyang Wu, et al.arXiv.org
Baichuan-M3 is a medical-enhanced large language model designed to shift from passive question-answering to active, clinical-grade decision support. It addresses limitations in open-ended consultations by modeling the systematic workflow of a physician, with key capabilities including proactive information acquisition, long-horizon reasoning, and adaptive…
Baichuan-M3 Team, :, Chengfeng Dou, Fan Yang, et al.arXiv.org
The paper introduces Region-to-Image Distillation (R2I), a method that internalizes the benefits of inference-time zooming into a single forward pass of a multimodal large language model (MLLM). R2I synthesizes fine-grained VQA data by zooming into micro-cropped regions, using strong teacher models to generate high-consensus question-answer pairs, and then…
Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, et al.arXiv.org
The paper introduces a method to improve spatial understanding in text-to-image generation using reward modeling. The authors construct the SpatialReward-Dataset, containing over 80,000 adversarial preference pairs, where each pair consists of an image correctly depicting complex spatial relationships and a perturbed image violating some relationships,…
Zhenyu Tang, Chaoran Feng, Yufan Deng, Jie Wu, et al.arXiv.org
This paper establishes a theoretical framework for analyzing entropy dynamics during reinforcement fine-tuning (RFT) of large language models. The authors derive a first-order expression for entropy change from a single logit update, showing that the direction of entropy change is determined by the update direction and a discriminator score S* = p_k(H +…
Shumin Wang, Yuexiang Xie, Wenhao Zhang, Yuchang Sun, et al.arXiv.org
The P1-VL technical report introduces a family of open-source vision-language models (VLMs) designed for advanced scientific reasoning, specifically targeting physics Olympiad problems. The models are trained exclusively via reinforcement learning (RL) using a curriculum that progressively increases problem difficulty and expands exploration space,…
Yun Luo, Futing Wang, Qianjia Cheng, Fangchen Yu, et al.arXiv.org
DeepImageSearch introduces a new paradigm for image retrieval, reformulating it as an agentic exploration task over visual histories. The authors construct DISBench, a benchmark with 122 queries across 57 users and 109,467 photos, built via a human-model collaborative pipeline that mines latent spatiotemporal associations. Queries are categorized into…
Chenlong Deng, Mengjie Deng, Junjie Wu, Dun Zeng, et al.arXiv.org
The paper introduces QRRanker, a listwise reranking framework that trains LLMs to score passage-query relevance using attention scores from selected Query-focused Retrieval (QR) heads, avoiding generation-based scoring and Likert-scale supervision. It uses a contrastive loss on continuous attention-derived scores, enabling training on arbitrary retrieval…
Yuqing Li, Jiangnan Li, Mo Yu, Guoxuan Ding, et al.arXiv.org
SLA2 is a sparse-linear attention method for diffusion models that addresses two limitations of the prior SLA approach: a heuristic split between sparse and linear branches, and a mismatch between SLA's output and the intended sparse-linear decomposition. SLA2 introduces a learnable router that dynamically selects which attention computations use sparse or…
Jintao Zhang, Haoxu Wang, Kai Jiang, Kaiwen Zheng, et al.arXiv.org
SkillOrchestra is a framework for skill-aware orchestration in compound AI systems, addressing limitations of existing routing approaches: input-level routers make coarse decisions, and RL-trained orchestrators are expensive and prone to routing collapse. Instead of learning a routing policy end-to-end, SkillOrchestra learns a reusable Skill Handbook from…
Jiayu Wang, Yifei Ming, Zixuan Ke, Shafiq Joty, et al.arXiv.org
USER is a unified and extensible system for real-world online policy learning in embodied AI, addressing the challenges of physical-world training such as lack of acceleration, reset, and replication. It treats robots as first-class hardware resources alongside GPUs via a unified hardware abstraction layer, enabling automatic discovery, management, and…
Hongzhi Zang, Shu'ang Yu, Hao Lin, Tianxing Zhou, et al.arXiv.org
HyTRec is a generative recommendation framework designed to efficiently model ultra-long user behavior sequences. It addresses the trade-off between linear attention's efficiency and softmax attention's precision by using a hybrid architecture: a linear attention branch (Temporal-Aware Delta Network, TADN) processes long-term history, while a softmax…
Lei Xin, Yuhao Zheng, Ke Cheng, Changjiang Jiang, et al.arXiv.org
This paper evaluates whether Sparse Autoencoders (SAEs) recover meaningful features from neural networks. In synthetic experiments with known ground-truth features, SAEs achieved 71% explained variance but recovered only 9% of true features, showing a disconnect between reconstruction fidelity and feature recovery. On real LLM activations, the authors…
Anton Korznikov, Andrey Galichin, Alexey Dontsov, Oleg Rogov, et al.arXiv.org
MolHIT is a molecular graph generation framework based on a Hierarchical Discrete Diffusion Model (HDDM) that generalizes discrete diffusion by adding mid-level states encoding chemical priors, and a Decoupled Atom Encoding (DAE) that splits atom types by chemical roles (aromaticity, formal charge). This addresses limitations in prior graph diffusion…
Hojung Jung, Rodrigo Hormazabal, Jaehyeong Jo, Youngrok Park, et al.DeepSeek
DualPath is an inference system that addresses the storage I/O bottleneck in multi-turn, agentic LLM inference under prefill-decode (PD) disaggregation. In such workloads, KV-Cache hit rates are high (≥95%), making KV-Cache loading from external storage the dominant performance factor. Existing systems load KV-Cache only into prefill engines, saturating…
Yongtong Wu, Shaoyuan Chen, Yinmin Zhong, Rilin Huang, et al.NVIDIA
PhyCritic is a multimodal critic model designed for physical AI tasks, addressing the lack of physics-aware evaluation in existing judge models. It uses a two-stage RLVR pipeline: first, a physical skill warmup stage with GRPO on physical QA pairs to enhance perception and reasoning; second, self-referential critic finetuning, where the critic generates…
Tianyi Xiong, Shihao Wang, Guilin Liu, Yi Dong, et al.arXiv.org
The paper introduces GENIUS, the first benchmark for evaluating Generative Fluid Intelligence (GFI) in Unified Multimodal Models (UMMs), distinguishing it from Crystallized Intelligence (CI). GFI is formalized into three primitives: Inducing Implicit Patterns, Executing Ad-hoc Constraints, and Adapting to Contextual Knowledge. The benchmark comprises 510…
Ruichuan An, Sihan Yang, Ziyu Guo, Wei Dai, et al.arXiv.org
The paper introduces ASID-1M, an open-source dataset of one million attribute-structured audiovisual instruction annotations, along with ASID-Verify, a multi-stage data curation pipeline for automatic annotation, verification, and refinement, and ASID-Captioner, a video understanding model trained via supervised fine-tuning on this dataset. The pipeline…
Yunheng Li, Hengrui Zhang, Meng-Hao Guo, Wenzhao Gao, et al.arXiv.org
BitDance is a scalable autoregressive image generation model that predicts binary visual tokens instead of codebook indices, scaling the vocabulary to 2^256 states. It introduces a binary diffusion head to sample from this large discrete space, and a next-patch diffusion method for parallel multi-token prediction. On ImageNet 256×256, BitDance achieves an…
Yuang Ai, Jiaming Han, Shaobin Zhuang, Weijia Mao, et al.arXiv.org
SkyReels-V4 is a unified multi-modal video foundation model for joint video-audio generation, inpainting, and editing. It uses a dual-stream Multimodal Diffusion Transformer (MMDiT) architecture with separate video and audio branches sharing a Multimodal Large Language Model (MLLM) text encoder. The model accepts text, images, video clips, masks, and audio…
Guibin Chen, Dixuan Lin, Jiangping Yang, Youqiang Zhang, et al.arXiv.org
The paper analyzes response length variation in Reinforcement Learning with Verifiable Rewards (RLVR) for large language models, focusing on the length bias in GRPO and GSPO objectives. GRPO averages token contributions per trajectory, causing shorter correct responses to receive larger gradient updates, while GSPO's sequence-level clipping and Clip-Higher…
Fanfan Liu, Youyang Yin, Peng Shi, Siqi Yang, et al.arXiv.org
The paper introduces daVinci-Agency, a data synthesis paradigm for training LLM agents on long-horizon tasks. It mines chains of semantically linked GitHub Pull Requests (PRs) to create training trajectories that capture task decomposition, long-term consistency, and iterative refinement. The method constructs task chains from up to five PRs, yielding…
Mohan Jiang, Dayuan Fu, Junhao Shi, Ji Zeng, et al.Anthropic
This paper presents the first application of crosscoders to cross-architecture model diffing, introducing Dedicated Feature Crosscoders (DFCs) to better isolate model-exclusive features. DFCs partition the feature space into model-exclusive and shared sets, overcoming the standard crosscoder's prior toward shared features. In synthetic toy models, DFCs…
Thomas Jiralerspong, Trenton BrickenTogether AI
ThunderAgent is a program-aware agentic inference system that improves throughput for multi-turn LLM agent workflows by abstracting them as agentic programs with metadata (ID, context length, tool environments, placement, phase, status). It introduces a program-aware scheduler with state-aware pausing and dynamic migration across GPU nodes to reduce KV…
Hao Kang, Ziyang Li, Xinyu Yang, Weili Xu, et al.Together AI
Aurora is a unified training-serving system that addresses limitations of conventional speculative decoding, which separates offline speculator training from online serving, causing deployment lag, delayed utility feedback, and domain-drift degradation. Aurora closes the loop by continuously learning a speculator from live inference traces, framing it as…
Junxiong Wang, Fengxiang Bie, Jisen Li, Zhongzhu Zhou, et al.Together AI
The paper introduces UPipe, a context parallelism technique that reduces activation memory in Transformer training by chunking attention heads. It builds on DeepSpeed-Ulysses, processing attention in stages with a subset of heads (U) at a time, reusing buffers to cut intermediate memory from O(H) to O(U). For Qwen3-32B, this reduces attention intermediate…
Ravi Ghadia, Maksim Abraham, S. Vorobyov, Max Ryabinin