Paper 2512.13281
VideoASMR-Bench: Can AI-Generated ASMR Videos Fool VLMs and Humans?
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 2
- GitHub
- 23 stars
01 In brief
Summary
VideoASMR-Bench is a new benchmark for evaluating AI-generated ASMR videos, focusing on fine-grained audio-visual perception and sensory immersion.
It includes 1,500 real ASMR videos from social media and 2,235 synthetic videos from nine video generation models (VGMs) under four settings.
The benchmark introduces an adversarial evaluation framework where VGMs try to fool video understanding models (VLMs) and VLMs try to detect fakes.
Results show that even state-of-the-art VLMs like Gemini-3-Pro fail to reliably detect AI-generated ASMR videos (accuracy 70.57 vs.
human 87.50), and the best VGM, Veo3.1-Fast, has only 12.54% of its videos correctly identified as fake.
Key findings include: watermarks act as detection shortcuts causing ~30-point performance drops; VLMs show a bias toward classifying videos as real (misclassifying up to 71% of fakes); auditory cues significantly improve detection; and the ASMR prompt suite consistently improves realism scores.
The benchmark is extensible, with open-sourced prompts and images for future scaling.
02 From the paper
Abstract
With AI-generated videos increasingly indistinguishable from reality, current benchmarks primarily focus on broad semantic alignment and basic physical consistency, offering limited discriminative power for evaluating them. To address this, we introduce VideoASMR-Bench, a benchmark based on Autonomous Sensory Meridian Response (ASMR) videos that emphasizes fine-grained audio-visual perception and sensory immersion. This benchmark aims to answer two key questions: (i) Are today's video understanding models (VLMs) sensitive enough to detect AI-generated ASMR videos by recognizing minor visual, physical, or auditory artifacts? (ii) Can today's video generation models (VGMs) produce convincing ASMR videos with immersive experiences? This benchmark comprises a diverse set of 1,500 high-quality real ASMR videos curated from social media, alongside 2,235 synthetic counterparts generated by nine VGMs. Additionally, we open-source an extensible suite of prompts and reference images, enabling the benchmark to scale dynamically with future video models. Moreover, we design an automatic understanding-generation evaluation framework between VGMs and VLMs, where VGMs aim to produce realistic fake videos to fool the VLMs, while the VLMs seek to detect them, forming an adversarial game between the two parties. Our evaluation on VideoASMR-Bench reveals that even state-of-the-art VLMs, such as Gemini-3-Pro, fail to reliably detect AI-generated ASMR videos. Meanwhile, current frontier video generation models can produce ASMR videos that are difficult for VLMs to distinguish from real ones, while humans can still identify them relatively easily.