Citation ranking / frozen snapshot
A snapshot-ranked view of scholarly impact across the collection, separate from Hugging Face popularity.
Qwen
Qwen3-VL is a state-of-the-art vision-language model family from the Qwen team, released on December 1, 2025. It supports interleaved contexts up to 256K tokens and comes in dense (2B/4B/8B/32B) and MoE (30B-A3B/235B-A22B) variants. Key architectural innovations include interleaved-MRoPE for balanced spatial-temporal encoding, DeepStack for multi-level ViT…
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, et al.Meta AI
DINOv3 is a self-supervised vision foundation model that scales to 7B parameters, trained on a curated dataset of 1.689 billion images (LVD-1689M) from Instagram, combined with ImageNet-1k. The model uses a ViT architecture with RoPE embeddings and constant hyperparameter schedules. A key contribution is Gram anchoring, a regularization technique that…
Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, et al.arXiv.org
InternVL3.5 is a new family of open-source multimodal models that improves versatility, reasoning, and efficiency over its predecessor InternVL3. Key innovations include Cascade Reinforcement Learning (Cascade RL), which combines offline RL (MPO) for stable convergence and online RL (GSPO) for refined alignment, boosting reasoning performance by up to…
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, et al.Qwen
Qwen-Image is a new image generation foundation model from the Qwen team, designed to excel in complex text rendering and precise image editing. It uses a comprehensive data pipeline and a progressive curriculum learning strategy to enhance text rendering, especially for Chinese. For editing, it employs a multi-task training paradigm with dual-encoding…
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, et al.Meta AI
SAM 3 is a unified model for promptable concept segmentation (PCS) in images and videos, accepting noun phrases, image exemplars, or both as prompts to detect, segment, and track all matching instances. It decouples recognition and localization via a presence head, improving detection accuracy. A data engine with human and AI verifiers produced 4M unique…
Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, et al.DeepSeek
DeepSeek-V3.2 introduces three key innovations to close the gap with closed-source models: DeepSeek Sparse Attention (DSA) reduces computational complexity from O(L^2) to O(Lk) while preserving long-context performance; a scalable reinforcement learning (RL) framework with a post-training compute budget exceeding 10% of pre-training cost enables…
DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, et al.arXiv.org
Depth Anything 3 (DA3) is a model that predicts spatially consistent geometry from any number of images, with or without known camera poses. It uses a single plain transformer (e.g., vanilla DINOv2) as backbone, with an input-adaptive cross-view self-attention mechanism and a dual-DPT head that jointly outputs depth and ray maps. A depth-ray representation…
Haotong Lin, Sili Chen, Junhao Liew, Donny Y. Chen, et al.Z.ai / GLM
GLM-4.5 is an open-source Mixture-of-Experts (MoE) large language model with 355B total and 32B activated parameters, developed by Zhipu AI and Tsinghua University. It features a hybrid reasoning method supporting both thinking and direct response modes. Trained on 23T tokens with multi-stage pre-training and mid-training (including repo-level code,…
GLM-4. 5 Team, :, Aohan Zeng, Xin Lv, et al.Qwen
Qwen3-Omni is a single multimodal model that achieves state-of-the-art performance across text, image, audio, and video without degradation relative to single-modal counterparts. It matches the performance of same-sized Qwen single-modal models and excels on audio tasks, achieving open-source SOTA on 32 of 36 audio benchmarks and overall SOTA on 22,…
Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, et al.Moonshot AI
Kimi K2.5 is an open-source multimodal agentic model that jointly optimizes text and vision through techniques including joint pre-training, zero-vision SFT, and joint reinforcement learning. It introduces Agent Swarm, a parallel agent orchestration framework using Parallel-Agent Reinforcement Learning (PARL) to decompose tasks into heterogeneous…
Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, et al.Z.ai / GLM
GLM-5, developed by Zhipu AI and Tsinghua University, is a next-generation foundation model that shifts from vibe coding to agentic engineering. It builds on the ARC (agentic, reasoning, coding) capabilities of its predecessor, GLM-4.7, and introduces DeepSeek Sparse Attention (DSA) to reduce training and inference costs while maintaining long-context…
GLM-5-Team, :, Aohan Zeng, Xin Lv, et al.OpenAI
The paper argues that language model hallucinations arise from statistical pressures during pretraining and persist due to misaligned evaluation metrics. The authors formalize hallucinations as errors in binary classification, showing that even with error-free training data, the cross-entropy objective leads to errors. They introduce the Is-It-Valid (IIV)…
Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin ZhangarXiv.org
The paper introduces ACE (Agentic Context Engineering), a framework for context adaptation in LLMs that treats contexts as evolving playbooks, accumulating and refining strategies through generation, reflection, and curation. ACE addresses two limitations of prior methods: brevity bias (over-compression of prompts) and context collapse (loss of detail…
Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, et al.arXiv.org
This survey provides a comprehensive overview of memory systems for foundation model-based AI agents, addressing the fragmentation in the field by proposing a unified taxonomy based on forms, functions, and dynamics. It defines agent memory, distinguishing it from LLM memory, RAG, and context engineering. The survey categorizes memory forms into…
Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, et al.arXiv.org
Seedream 4.0 is a multimodal image generation system by ByteDance Seed that unifies text-to-image (T2I) synthesis, image editing, and multi-image composition in a single framework. It uses an efficient diffusion transformer (DiT) with a high-compression VAE, reducing image tokens and enabling native 1K-4K resolution generation. The model is pretrained on…
Team Seedream, :, Yunpeng Chen, Yu Gao, et al.arXiv.org
The paper introduces Representation Autoencoders (RAEs), which replace the traditional VAE in latent diffusion models with a frozen pretrained representation encoder (e.g., DINOv2, SigLIP2, MAE) paired with a trained decoder. RAEs provide both high-quality reconstructions and semantically rich latent spaces, outperforming SD-VAE in reconstruction FID…
Boyang Zheng, Nanye Ma, Shengbang Tong, Saining XiearXiv.org
Z-Image is a 6B-parameter image generation foundation model from Alibaba Group, built on a Scalable Single-Stream Diffusion Transformer (S3-DiT). It challenges the 'scale-at-all-costs' paradigm by optimizing data infrastructure, architecture, training, and inference. The full training workflow costs 314K H800 GPU hours (~$628K). Z-Image-Turbo, a distilled…
Z-Image Team, Huanqia Cai, Sihan Cao, Ruoyi Du, et al.Meta AI
SAM 3D is a generative model for 3D object reconstruction from a single image, predicting geometry, texture, and layout. It excels in natural images with occlusion and clutter, using a human- and model-in-the-loop pipeline to create large-scale 3D annotation data. The model uses a multi-stage training framework: synthetic pretraining on 2.7M meshes…
SAM 3D Team, Xingyu Chen, Fu-Jen Chu, Pierre Gleize, et al.Google DeepMind
This paper investigates whether generative video models, like Veo 3, can act as zero-shot learners and reasoners for general-purpose vision tasks, similar to how LLMs transformed NLP. The authors analyzed 18,384 generated videos across 62 qualitative and 7 quantitative tasks, finding that Veo 3 can solve tasks it wasn't explicitly trained for, including…
Thaddäus Wiedemer, Yuxuan Li, Paul Vicol, Shixiang Shane Gu, et al.NVIDIA
LONGLIVE is a frame-level autoregressive (AR) framework for real-time, interactive long video generation, addressing efficiency and quality challenges in diffusion and AR models. It introduces KV-recache to refresh cached states with new prompts for smooth, adherent prompt switches; streaming long tuning to enable train-long-test-long alignment; and…
Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, et al.arXiv.org
This survey formalizes Agentic Reinforcement Learning (Agentic RL), a paradigm where LLMs are treated as learnable policies in sequential decision-making loops, contrasting it with conventional single-step LLM RL. It proposes a twofold taxonomy: one based on core agentic capabilities (planning, tool use, memory, reasoning, self-improvement, perception) and…
Guibin Zhang, Hejia Geng, Xiaohang Yu, Zhenfei Yin, et al.arXiv.org
SKILLSBENCH is a benchmark for evaluating Agent Skills, which are structured packages of procedural knowledge that augment LLM agents at inference time. The benchmark contains 87 tasks across 8 domains, each with curated Skills and deterministic verifiers. In an evaluation of 18 model–harness configurations, curated Skills raised the average pass rate from…
Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, et al.arXiv.org
R-Zero is a fully autonomous framework for training reasoning LLMs without any pre-existing tasks or labels. It initializes two models from a single base LLM: a Challenger, trained via GRPO to generate questions at the edge of the Solver's capability (using an uncertainty reward based on self-consistency, plus repetition and format penalties), and a…
Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, et al.Qwen
The report introduces Qwen3-VL-Embedding and Qwen3-VL-Reranker, a unified framework for multimodal retrieval built on the Qwen3-VL foundation model. The embedding model uses a multi-stage training pipeline (contrastive pre-training, multi-task contrastive learning, and reranker distillation) to produce high-dimensional vectors, supporting Matryoshka…
Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, et al.arXiv.org
Seed Diffusion Preview is a large-scale discrete-state diffusion language model developed by ByteDance Seed and Tsinghua AIR, focused on code generation. It achieves an inference speed of 2,146 tokens per second on H20 GPUs, significantly faster than contemporary models like Mercury Coder and Gemini Diffusion, while maintaining competitive performance on…
Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, et al.arXiv.org
UI-TARS-2 is a native GUI-centered agent model developed by ByteDance Seed, designed to handle both structured computer-use tasks and dynamic game environments. It addresses challenges in data scalability, multi-turn reinforcement learning (RL), GUI-only operation limits, and environment stability through a systematic methodology: a data flywheel for…
Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, et al.arXiv.org
This survey reviews recent advances in Reinforcement Learning (RL) for Large Reasoning Models (LRMs), focusing on how RL transforms LLMs into LRMs, particularly since DeepSeek-R1. It covers foundational components: reward design (verifiable, generative, dense, unsupervised, and shaping), policy optimization (critic-based, critic-free, off-policy, and…
Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, et al.DeepSeek
DeepSeek-OCR is a vision-language model that explores optical compression of long contexts by mapping text to images and decoding them back to text. It consists of DeepEncoder, a novel vision encoder with low activation and high compression, and a DeepSeek3B-MoE decoder. On the Fox benchmark, it achieves 97% OCR precision at compression ratios under 10x,…
Haoran Wei, Yaofeng Sun, Yukun LiarXiv.org
Self-Forcing++ is a method for long-horizon video generation that extends autoregressive diffusion models beyond the training horizon of their teacher models. It addresses quality degradation from error accumulation by generating long self-rollouts (up to 100 seconds), re-injecting noise into these degraded sequences (backward noise initialization), and…
Justin Cui, Jie Wu, Ming Li, Tao Yang, et al.arXiv.org
This survey reviews the emerging paradigm of self-evolving AI agents, which aim to bridge static foundation models with lifelong adaptability. It introduces a unified framework with four components: system inputs, agent system, environment, and optimizers, and proposes 'Three Laws' (Endure, Excel, Evolve) to guide safe and effective evolution. The survey…
Jinyuan Fang, Yanwen Peng, Xi Zhang, Yingxu Wang, et al.Meta AI
Deep Think with Confidence (DeepConf) is a test-time method that improves LLM reasoning efficiency and accuracy by using model-internal confidence signals to filter low-quality reasoning traces. It addresses the diminishing returns and high computational overhead of self-consistency with majority voting. DeepConf operates in offline mode (filtering…
Yichao Fu, Xuewei Wang, Yuandong Tian, Jiawei ZhaoarXiv.org
SKILLRL is a framework for LLM agents that improves policy learning by distilling raw interaction trajectories into a hierarchical skill library (SKILLBANK) and co-evolving it with the agent's policy during reinforcement learning. It uses a teacher model to extract general and task-specific skills from both successful and failed episodes, achieving 10-20x…
Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, et al.arXiv.org
The paper introduces GUI-Owl, a foundational GUI agent model built on Qwen2.5-VL, and Mobile-Agent-v3, a multi-agent framework. GUI-Owl-7B achieves state-of-the-art scores among open-source models, including 66.4 on AndroidWorld and 34.9 on OSWorld-Verified, while Mobile-Agent-v3 further improves these to 73.3 and 37.7, respectively. Key innovations…
Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, et al.arXiv.org
This paper systematically investigates on-policy distillation (OPD) for large language models, identifying conditions for success, the token-level mechanism, and practical remedies. The authors find that OPD succeeds only when the student and teacher share compatible thinking patterns and the teacher offers genuinely new knowledge beyond the student's…
Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, et al.arXiv.org
SimpleTIR is a plug-and-play algorithm that stabilizes multi-turn Tool-Integrated Reasoning (TIR) training under the Zero RL paradigm. The authors identify that training instability and gradient explosions stem from distributional drift caused by external tool feedback, which leads to the generation of low-probability tokens that compound over turns. They…
Zhenghai Xue, Longtao Zheng, Qian Liu, Yingru Li, et al.arXiv.org
LTX-2 is an open-source text-to-audio+video (T2AV) foundation model that jointly generates synchronized video and audio from text. It uses an asymmetric dual-stream transformer with a 14B-parameter video stream and a 5B-parameter audio stream, connected via bidirectional cross-attention layers with temporal positional embeddings and cross-modality AdaLN.…
Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, et al.arXiv.org
SimpleVLA-RL is an efficient reinforcement learning (RL) framework for Vision-Language-Action (VLA) models, built on veRL, that addresses data scarcity and poor generalization in robotic manipulation. It uses interactive trajectory sampling, parallel multi-environment rendering, and outcome-based rewards (1 for success, 0 for failure) with GRPO, enhanced…
Haozhan Li, Yuxin Zuo, Jiale Yu, Yuhao Zhang, et al.AAAI Conference on Artificial Intelligence
VLA-Adapter is a novel paradigm for Vision-Language-Action (VLA) models that reduces reliance on large-scale VLMs and extensive pre-training. It systematically analyzes which vision-language (VL) conditions are essential for bridging perception to action, finding that middle-layer raw features and deep-layer ActionQuery features are most effective, and…
Yihao Wang, Pengxiang Ding, Lingxiao Li, Can Cui, et al.arXiv.org
MiniCPM-V 4.5 is an 8B-parameter multimodal large language model (MLLM) designed for high efficiency and strong performance. It introduces three core improvements: a unified 3D-Resampler architecture that compresses images and videos into compact token sequences (up to 16x for images and an additional 6x for videos), a unified learning paradigm for…
Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, et al.arXiv.org
LLaDA2.0 introduces a family of discrete diffusion language models (dLLMs) scaled up to 100B parameters by converting pre-trained auto-regressive (AR) models, avoiding costly training from scratch. The conversion uses a three-phase Warmup-Stable-Decay (WSD) continual pre-training strategy: progressively increasing block size in block diffusion (warmup),…
Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, et al.arXiv.org
The paper introduces Thyme (Think Beyond Images), a paradigm for multimodal large language models (MLLMs) that autonomously generates and executes code for image manipulations (cropping, rotation, contrast enhancement) and complex computations, going beyond existing 'think with images' approaches. Thyme uses a two-stage training: Supervised Fine-Tuning…
Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, et al.Moonshot AI
Kimi Linear is a hybrid linear attention architecture that outperforms full attention under fair comparisons across short-context, long-context, and reinforcement learning (RL) scaling regimes. Its core, Kimi Delta Attention (KDA), extends Gated DeltaNet with a finer-grained, channel-wise gating mechanism, improving memory control and enabling efficient…
Kimi Team, Yu Zhang, Zongyu Lin, Xingcheng Yao, et al.NVIDIA
The paper identifies a flaw in applying Group Relative Policy Optimization (GRPO) to multi-reward reinforcement learning: normalizing the summed reward causes distinct reward combinations to collapse into identical advantage values, reducing training signal resolution and causing suboptimal convergence or early failure. To address this, the authors propose…
Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, et al.NVIDIA
Fast-dLLM v2 is a block diffusion language model that adapts pretrained autoregressive LLMs (Qwen2.5-Instruct 1.5B and 7B) for parallel text generation. It requires only ~1B tokens of fine-tuning, a 500x reduction compared to full-attention diffusion models like Dream (580B tokens). The method uses block-wise diffusion with complementary masking and a…
Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, et al.arXiv.org
Genie Envisioner (GE) is a unified world foundation platform for robotic manipulation, integrating policy learning, evaluation, and simulation within a video-generative framework. GE-Base, an instruction-conditioned multi-view video diffusion model, is pretrained on ~3,000 hours of data from one million real-world episodes (AgiBot-World-Beta) to capture…
Yue Liao, Pengfei Zhou, Siyuan Huang, Donglin Yang, et al.arXiv.org
This paper analyzes the generalization gap between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) for Large Language Models. The authors show that the SFT gradient can be interpreted as a policy gradient with an implicit, sparse reward that is inversely proportional to the model's token probability, leading to unstable updates and poor…
Yongliang Wu, Yizhou Zhou, Zhou Ziheng, Yingzhe Peng, et al.arXiv.org
LightMem is a lightweight and efficient memory-augmented generation system for Large Language Models (LLMs), presented at ICLR 2026. It addresses inefficiencies in existing memory systems by drawing inspiration from the Atkinson-Shiffrin human memory model, organizing memory into three stages: sensory memory (pre-compression and topic segmentation),…
Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, et al.arXiv.org
Emu3.5 is a large-scale multimodal world model from BAAI that natively predicts the next state across vision and language. It is pre-trained end-to-end with a unified next-token prediction objective on over 10 trillion tokens from interleaved vision-language data, primarily derived from internet videos. The model accepts and generates interleaved…
Yufeng Cui, Honghao Chen, Haoge Deng, Xu Huang, et al.OpenAI
This paper introduces GDPval, a benchmark for evaluating AI models on real-world, economically valuable tasks. It covers 44 occupations across the top 9 U.S. GDP sectors, with tasks created by industry experts averaging 14 years of experience. The benchmark includes 1,320 tasks in the full set and a 220-task gold subset, graded via human expert pairwise…
Tejal Patwardhan, Rachel Dias, Elizabeth Proehl, Grace Kim, et al.arXiv.org
The paper introduces Ouro, a family of pre-trained Looped Language Models (LoopLM) that integrate iterative computation in latent space during pre-training, using an entropy-regularized objective for adaptive depth allocation. Trained on 7.7T tokens, Ouro 1.4B and 2.6B models match or exceed the performance of 4B and 8B standard transformers, achieving…
Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, et al.Qwen
WebWatcher is a multimodal deep research agent introduced by Alibaba's Tongyi Lab to address the limitations of text-centric web agents by integrating visual information and complex reasoning. It uses high-quality synthetic multimodal trajectories for cold-start training, multiple tools (web search, image search, webpage visit, code interpreter, OCR), and…
Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, et al.arXiv.org
Mini-o3 is a system for visual search that scales up tool-based interactions and reasoning patterns, achieving state-of-the-art performance on challenging tasks. It addresses limitations of existing open-source VLMs, which show monotonous reasoning and limited interaction turns. The approach includes three key components: constructing the Visual Probe…
Xin Lai, Junyi Li, Wei Li, Tao Liu, et al.arXiv.org
WorldPlay is a real-time interactive world model that generates streaming 720p video at 24 FPS while maintaining long-term geometric consistency. It addresses the trade-off between speed and memory in existing methods. The model uses three key components: Dual Action Representation combining discrete keys and continuous camera poses for robust control;…
Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, et al.arXiv.org
ReSum is a plug-and-play paradigm that enables LLM-based web agents to perform unbounded exploration by periodically invoking an external summarization tool to condense interaction histories into compact summaries, addressing the conflict between extensive exploration and limited context windows. The authors developed ReSumTool-30B, a specialized summary…
Xixi Wu, Kuan Li, Yida Zhao, Liwen Zhang, et al.arXiv.org
The paper introduces Spatial Forcing (SF), a method to enhance the spatial awareness of Vision-Language-Action (VLA) models without explicit 3D inputs. VLA models, built on 2D-pretrained VLMs, lack 3D understanding, limiting their robotic manipulation performance. Existing solutions using depth sensors or point clouds face issues like sensor noise and data…
Fuhao Li, Wenxuan Song, Han Zhao, Jingbo Wang, et al.arXiv.org
MemSkill reframes LLM agent memory operations as learnable, evolvable memory skills. It maintains a shared skill bank where each skill provides structured guidance (purpose, when to use, how to apply, constraints) for extracting, consolidating, or revising memories. A controller, trained with reinforcement learning (PPO), selects a Top-K set of relevant…
Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, et al.arXiv.org
Vision-SR1 is a three-stage self-rewarding reinforcement learning method for vision-language models (VLMs) that improves visual reasoning without external visual supervision. It decomposes VLM reasoning into visual perception and language reasoning, prompting the model to first generate a self-contained visual description, then re-prompting it to answer…
Zongxia Li, Wenhao Yu, Chengsong Huang, Zhenwen Liang, et al.arXiv.org
The paper introduces Pref-GRPO, a reinforcement learning method for text-to-image (T2I) generation that replaces pointwise reward score maximization with pairwise preference fitting to address reward hacking. It identifies the root cause of reward hacking as an 'illusory advantage' arising from normalizing tightly clustered pointwise scores by a small…
Yibin Wang, Zhimin Li, Yuhang Zang, Yujie Zhou, et al.arXiv.org
MCP-Bench is a benchmark for evaluating LLM agents on realistic, multi-step tool-use tasks via the Model Context Protocol (MCP). It connects agents to 28 live MCP servers with 250 tools across domains like finance, travel, and science, enabling complex intra- and cross-server workflows. Tasks are generated via an LLM-based synthesis pipeline that discovers…
Zhenting Wang, Qi Chang, Hemani Patel, Shashank Biju, et al.Qwen
The Qwen3-TTS technical report introduces a family of multilingual, controllable, robust, and streaming text-to-speech models. Trained on over 5 million hours of speech data across 10 languages, Qwen3-TTS supports 3-second voice cloning, description-based voice design, and fine-grained control. It uses a dual-track LM architecture with two tokenizers:…
Hangrui Hu, Xinfa Zhu, Ting He, Dake Guo, et al.arXiv.org
The paper introduces Memento, a learning paradigm for LLM agents that enables continual adaptation without fine-tuning the underlying LLM. It formalizes a Memory-augmented Markov Decision Process (M-MDP) with a neural case-selection policy, storing past experiences in an episodic case bank. The policy is updated via online soft Q-learning, with retrieval…
Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, et al.arXiv.org
LingBot-World is an open-source world simulator built from video generation, offering high-fidelity dynamics across diverse environments, minute-level long-term memory, and real-time interactivity with sub-second latency. It uses a three-stage training pipeline: pre-training on a 14B-parameter Wan2.2 model, middle-training with a mixture-of-experts (MoE)…
Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, et al.arXiv.org
The paper introduces Generalized On-Policy Distillation (G-OPD), a framework that extends standard on-policy distillation (OPD) by adding a flexible reference model and a reward scaling factor λ. The authors show that OPD is a special case of dense KL-constrained RL with equal reward and KL weights. In G-OPD, λ controls the relative weight of the reward…
Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, et al.arXiv.org
Qwen3.5-Omni is a fully omnimodal large language model that scales to hundreds of billions of parameters and supports a 256k context length. It is pretrained on a massive dataset including over 100 million hours of audio-visual content. The model uses a Thinker-Talker architecture with Hybrid-Attention Mixture-of-Experts (MoE) for both components, enabling…
Qwen TeamarXiv.org
Lumina-DiMOO is an open-source, 8B-parameter unified multi-modal model based on a fully discrete diffusion architecture, built on the LLaDA-Base dLLM. It supports text-to-image generation at arbitrary resolutions, image-to-image tasks (editing, style transfer, subject-driven, controllable, multi-view), and image understanding. The model uses a unified…
Yi Xin, Qi Qin, Siqi Luo, Kaiwen Zhu, et al.arXiv.org
MinerU2.5 is a 1.2B-parameter vision-language model for document parsing that achieves state-of-the-art accuracy with high efficiency. It uses a decoupled, two-stage strategy: first, global layout analysis on a downsampled 1036x1036 image; second, targeted content recognition on native-resolution crops guided by the layout. The model uses a 675M NaViT…
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, et al.NVIDIA
SANA-Video is a small diffusion model for efficient, high-resolution (up to 720×1280) and minute-long video generation, deployable on RTX 5090 GPUs. It uses a Linear DiT with linear attention (O(N) complexity) and a constant-memory KV cache for block linear attention, enabling long videos with fixed memory. Training cost is 12 days on 64 H100 GPUs (1% of…
Junsong Chen, Yuyang Zhao, Jincheng Yu, Ruihang Chu, et al.arXiv.org
Ovis2.5 is a multimodal large language model from Alibaba Group, succeeding Ovis2. It introduces native-resolution visual perception using a NaViT vision transformer, which processes images at their original resolutions to preserve fine details and global layout, crucial for complex charts. The model also adds an optional 'thinking mode' for deep…
Shiyin Lu, Yang Li, Yu Xia, Yuwei Hu, et al.arXiv.org
VERLTOOL is a unified, modular framework for Agentic Reinforcement Learning with Tool use (ARLT), addressing fragmentation, synchronous execution bottlenecks, and limited extensibility in existing ARLT codebases. It builds on VeRL, providing upstream alignment, a standardized tool server API supporting diverse tools (code execution, search, SQL, vision,…
Dongfu Jiang, Yi Lu, Zhuofeng Li, Zhiheng Lyu, et al.arXiv.org
Seedance 2.0, released by ByteDance in early February 2026, is a native multimodal audio-video generation model that supports text, image, audio, and video inputs. It generates 4-15 second clips at 480p/720p, with a Fast version for low latency. The model excels in real-world complexity, multimodal reference and editing, high-fidelity binaural audio, and…
Team Seedance, De Chen, Liyang Chen, Xin Chen, et al.arXiv.org
Trace2Skill is a framework that distills agent execution trajectories into portable skills by analyzing many traces in parallel and consolidating recurring lessons into a single skill directory. It supports both deepening existing human-written skills and creating skills from weak LLM-generated drafts. The pipeline has three stages: trajectory generation…
Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, et al.Anthropic
This paper investigates whether poisoning attacks on large language models (LLMs) require a constant number of poisoned samples regardless of dataset size, rather than a fixed percentage. The authors conducted the largest pretraining poisoning experiments to date, training models from 600M to 13B parameters on Chinchilla-optimal datasets (6B to 260B…
Alexandra Souly, Javier Rando, Ed Chapman, Xander Davies, et al.arXiv.org
The paper introduces DR Tulu-8B, the first fully open model trained end-to-end for long-form deep research tasks, using a new method called Reinforcement Learning with Evolving Rubrics (RLER). RLER constructs and maintains rubrics that co-evolve with the policy model during training, incorporating newly explored information from search and contrasting…
Rulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, et al.arXiv.org
The paper introduces TraceRL, a trajectory-aware reinforcement learning framework for diffusion language models (DLMs) that aligns training objectives with inference trajectories, improving reasoning performance on math and coding tasks. It includes a diffusion-based value model for training stability and supports both full-attention and block-attention…
Yinjie Wang, Ling Yang, Bowen Li, Ye Tian, et al.arXiv.org
WebExplorer introduces a data synthesis framework for training long-horizon web agents. It uses model-based exploration, where LLMs iteratively search and browse from a seed entity to construct an information space, and iterative long-to-short query evolution, which removes salient clues and adds obfuscation to increase query difficulty. This produces the…
Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, et al.arXiv.org
StreamingVLM is a framework for real-time understanding of infinite video streams, addressing the limitations of full attention (quadratic cost, poor long-video performance) and sliding window methods (coherence loss or high latency). It maintains a compact KV cache with attention sinks, a short vision window, and a long text window, using contiguous RoPE…
Ruyi Xu, Guangxuan Xiao, Yukang Chen, Liuning He, et al.Together AI
The paper introduces TTT-Discover, a method that performs reinforcement learning at test time to solve scientific discovery problems. Unlike prior test-time scaling methods that use a frozen LLM for search, TTT-Discover continues to train the LLM on the specific test problem, using an entropic objective and PUCT-based state reuse to prioritize the most…
Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, et al.arXiv.org
PaddleOCR-VL is a state-of-the-art, resource-efficient document parsing solution from Baidu's PaddlePaddle team. It uses a two-stage architecture: PP-DocLayoutV2 for layout analysis (detection, classification, reading order) and PaddleOCR-VL-0.9B, a compact vision-language model combining a NaViT-style dynamic resolution visual encoder with the…
Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, et al.arXiv.org
This book provides a comprehensive, mathematically grounded introduction to diffusion models, tracing their origins from three foundational perspectives: variational (VAEs to DDPMs), score-based (EBMs to NCSN and Score SDEs), and flow-based (Normalizing Flows to Flow Matching). It unifies these views through the Fokker-Planck equation and a common…
Chieh-Hsin Lai, Yang Song, Dongjun Kim, Yuki Mitsufuji, et al.arXiv.org
UniVideo is a unified multimodal framework for video understanding, generation, and editing, presented at ICLR 2026. It uses a dual-stream architecture combining a frozen Multimodal Large Language Model (MLLM, Qwen2.5VL-7B) for understanding and a Multimodal DiT (MMDiT, HunyuanVideo-T2V-13B) for generation, connected via a trainable MLP connector. This…
Cong Wei, Quande Liu, Zixuan Ye, Qiulin Wang, et al.Together AI
Mamba-3 introduces three methodological improvements to state space models (SSMs) for better quality, capability, and inference efficiency. First, exponential-trapezoidal discretization generalizes prior heuristics, enabling a more expressive recurrence that can replace the short causal convolution. Second, complex-valued state transitions, implemented via…
Aakash Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang, et al.arXiv.org
OpenClaw-RL is a framework that enables online reinforcement learning for LLM-based agents by extracting training signals from the next-state signals that follow each agent action (e.g., user replies, tool outputs, GUI changes). It uses a server–client architecture where the RL server hosts the policy behind an inference API and user terminals stream…
Yinjie Wang, Xuyang Chen, Xiaolong Jin, Mengdi Wang, et al.arXiv.org
The paper introduces Chain-of-Agents (CoA), a paradigm enabling a single LLM to perform multi-agent problem-solving end-to-end by dynamically activating role-playing and tool agents. To train these Agent Foundation Models (AFMs), the authors propose multi-agent distillation, which converts trajectories from state-of-the-art multi-agent systems (e.g.,…
Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, et al.arXiv.org
WebResearcher is a framework for deep-research agents that addresses the limitations of mono-contextual approaches, which suffer from context suffocation and noise contamination. It introduces IterResearch, an iterative paradigm that reformulates deep research as a Markov Decision Process, where agents periodically consolidate findings into evolving…
Zile Qiao, Guoxin Chen, Xuanzhong Chen, Donglei Yu, et al.DeepSeek
The paper introduces Manifold-Constrained Hyper-Connections (mHC), a framework that addresses training instability and scalability issues in Hyper-Connections (HC) by projecting residual connection matrices onto the Birkhoff polytope (doubly stochastic matrices) using the Sinkhorn-Knopp algorithm. This restores the identity mapping property, ensuring norm…
Zhenda Xie, Yixuan Wei, Huanqi Cao, Chenggang Zhao, et al.arXiv.org
AgentFold is a new web agent paradigm that addresses the trade-off between context saturation in ReAct-based agents and information loss from fixed summarization. It treats context as a dynamic workspace with Multi-Scale State Summaries and a Latest Interaction, using a 'folding' operation with two modes: Granular Condensation (preserving fine details) and…
Rui Ye, Zhongwang Zhang, Kuan Li, Huifeng Yin, et al.arXiv.org
This survey provides the first comprehensive review of 3D and 4D world modeling, addressing the lack of standardized definitions and the fragmented literature that often focuses on 2D generative methods. It establishes precise definitions and a hierarchical taxonomy categorizing methods into video-based (VideoGen), occupancy-based (OccGen), and LiDAR-based…
Lingdong Kong, Yu Yang, Jianbiao Mei, Youquan Liu, et al.arXiv.org
The paper introduces Recursive Language Models (RLMs), an inference-time framework that treats arbitrarily long prompts as part of an external environment, allowing an LLM to programmatically examine, decompose, and recursively call itself over prompt snippets. RLMs use a REPL environment where the prompt is stored as a variable, and the model writes code…
Alex L. Zhang, Tim Kraska, Omar KhattabarXiv.org
The paper introduces M3-Agent, a multimodal agent framework with long-term memory that processes real-time video and audio to build episodic and semantic memories, organized in an entity-centric multimodal graph. It uses reinforcement learning for multi-turn reasoning and iterative memory retrieval. The authors also present M3-Bench, a long-video question…
Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, et al.arXiv.org
The paper introduces Visual Spatial Tuning (VST), a framework to enhance the spatial perception and reasoning abilities of Vision-Language Models (VLMs) without adding specialized 3D encoders. VST comprises two datasets: VST-P, with 4.1 million samples across 19 tasks covering single-image, multi-image, and video scenarios, and VST-R, with 135K samples for…
Rui Yang, Ziyu Zhu, Yanwei Li, Jingjia Huang, et al.The Web Conference
DeepAgent is an end-to-end deep reasoning agent that unifies autonomous thinking, tool discovery, and action execution within a single reasoning process, addressing limitations of predefined workflow-based agents. It introduces an autonomous memory folding mechanism that compresses interaction history into structured episodic, working, and tool memories,…
Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Guanting Dong, et al.arXiv.org
DFlash is a speculative decoding framework that uses a lightweight block diffusion model for parallel drafting, addressing the sequential bottleneck of autoregressive LLM decoding. It extracts hidden features from the target LLM and injects them into every draft layer's KV cache, conditioning the draft model to predict future token blocks in parallel. This…
Jian Chen, Yesheng Liang, Zhijian LiuDeepSeek
The paper introduces Engram, a conditional memory module that augments Mixture-of-Experts (MoE) large language models with static N-gram lookup tables, providing a complementary sparsity axis to conditional computation. Engram uses hashed N-gram embeddings with tokenizer compression, multi-head hashing, and context-aware gating to retrieve static knowledge…
Xin Cheng, Rui Tian, Wangding Zeng, Damai Dai, et al.arXiv.org
MiroThinker v1.0 is an open-source research agent that introduces interaction scaling as a third performance dimension, alongside model size and context length. It trains models via reinforcement learning to handle deeper and more frequent agent–environment interactions, using a 256K context window and up to 600 tool calls per task. The 72B variant…
MiroMind Team, Song Bai, Lidong Bing, Carson Chen, et al.arXiv.org
AgentGym-RL is a unified, modular reinforcement learning framework for training LLM agents in multi-turn, long-horizon decision-making tasks, without requiring supervised fine-tuning. It supports diverse environments (web navigation, deep search, digital games, embodied tasks, scientific tasks) and mainstream RL algorithms (PPO, GRPO, REINFORCE++, RLOO).…
Zhiheng Xi, Jixuan Huang, Chenyang Liao, Baodai Huang, et al.arXiv.org
This paper investigates why self-distillation can degrade the mathematical reasoning performance of LLMs, despite often improving other domains. The authors find that self-distillation suppresses 'epistemic verbalization'—the expression of uncertainty during reasoning—which is crucial for robust problem-solving. Through controlled experiments, they show…
Jeonghye Kim, Xufang Luo, Minbeom Kim, Sangmook Lee, et al.DeepSeek
DeepSeekMath-V2 is a large language model for natural-language theorem proving, built on DeepSeek-V3.2-Exp-Base, that achieves self-verifiable mathematical reasoning. The authors argue that final-answer rewards are insufficient because correct answers do not guarantee correct reasoning and are inapplicable to theorem proving. They train a verifier using…
Zhihong Shao, Yuxiang Luo, Chengda Lu, Z. Z. Ren, et al.arXiv.org
WebSailor-V2 is a post-training pipeline for open-source web agents, comprising data construction (SailorFog-QA-V2), SFT, and RL. The dataset uses a dense knowledge graph with cyclic structures and diverse uncertainty types beyond obfuscation. Training employs a dual-environment RL framework: a high-fidelity simulator for rapid iteration and a managed…
Kuan Li, Zhongwang Zhang, Huifeng Yin, Rui Ye, et al.arXiv.org
Rex-Omni is a 3B-parameter multimodal large language model (MLLM) that unifies object detection and other visual perception tasks under a next-point prediction framework. It uses quantized relative coordinates represented by 1,000 special tokens, reducing token usage per bounding box to four. Training combines supervised fine-tuning on 22 million samples…
Qing Jiang, Junan Huo, Xingyu Chen, Yuda Xiong, et al.arXiv.org
TreePO is a reinforcement learning framework for large language models that replaces standard independent trajectory sampling with a heuristic tree-based rollout mechanism. It models sequence generation as a tree search with dynamic branching and fixed-length segment decoding, leveraging shared prefixes via KV-cache reuse to reduce compute. The method…
Yizhi Li, Qingshui Gu, Zhoufutu Wen, Ziniu Li, et al.