When Slower Isn't Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning

Reasoning models have attracted increasing attention for their ability to tackle complex tasks, embodying the System II (slow thinking) paradigm in contrast to System I (fast, intuitive responses). Yet a key question remains: Does slower reasoning necessarily lead to more truthful answers?

Our findings suggest otherwise. We conduct the first systematic study of the inverse scaling law in slow-thinking paradigms for multimodal reasoning. We find that when confronted with incomplete or misleading visual inputs, slow-thinking models are more prone to fabricating plausible yet false details to justify dishonest reasoning.

To analyze this behavior, we construct a 5,000-sample hierarchical prompt dataset annotated by 50 human participants. The prompts progressively increase in complexity, revealing a consistent pattern: slower reasoning models tend to follow depth-first search (DFS) thinking, persistently exploring flawed premises, while faster chat models favor breadth-first search (BFS) inference, showing greater caution under uncertainty.

Contributions:

• First and foremost, human-in-the-loop. We introduce TruthfulVQA, the first large-scale multimodal truthfulness benchmark built with rigorous human-in-the-loop verification. Over 5k visually misleading images were collected and annotated by 50 professional annotators, each independently reviewed by five annotators on a case-by-case basis.
• Hierarchical prompt design for deep truthfulness evaluation. We propose a three-tier human-written prompt that systematically probes models across increasing levels of reasoning complexity, enabling finer-grained diagnosis of dishonesty and misinformation vulnerabilities.
• Revealing slow vs. fast thinking pitfalls in multimodal reasoning. We conduct the first comprehensive analysis comparing depth-first (slow thinking) reasoning models and breadth-first (fast thinking) chat models under adversarial visual conditions. Reasoning models are significantly more prone to factual dishonesty in complex visual tasks.
• TruthfulJudge — Reliable Human-Centric Evaluation Pipeline. We design TruthfulJudge, a reliable evaluation pipeline with specialized judge model (ECE=0.12, Cohen's κ=0.79, 88.4% judge accuracy).