Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs

Zhenhong Sun1 Hanqing Zhao2 Yatao Bian3 Rongcheng Tu2 Liuyue Xie4
Xu Zhang4 Jue Wang4 Davide Modolo4 Daoyi Dong5 Dacheng Tao2
1Australian National University 2Nanyang Technological University 3National University of Singapore
4Amazon 5University of Technology Sydney
Paper Code arXiv BibTeX
Overview of the Evolutionary Decoding framework.
Evolutionary Decoding intervenes in block-wise diffusion decoding with step-wise selection and block-wise mutation.

Abstract

Diffusion large language models offer efficient generation through block-wise progressive unmasking, but local token confidence can become misaligned with global mathematical correctness. We identify a diffusion confidence trap with two regimes: sampling-sensitive failures, where correct paths exist but are unstable, and sampling-consistent failures, where repeated sampling converges to confident but incorrect continuations.

Evolutionary Decoding is a training-free test-time scaling framework that reshapes the decoding trajectory. Step-wise selection preserves useful numerical-symbolic signals and suppresses repetitive patterns, while block-wise mutation introduces structured alternatives to escape incorrect high-confidence basins.

Confidence Trap

Sampling failure regimes.
Repeated runs reveal sampling-sensitive and sampling-consistent failures.
High-confidence repetitive trajectory.
A representative high-confidence repetitive trajectory.

Method

Step-wise Selection

Promotes uncertain but reasoning-critical digits and symbols during progressive unmasking.

Block-wise Mutation

Adds numerical, symbolic, mixed, and neutral branches before a block collapses into a fixed wrong direction.

Training-free

Uses frozen LLaDA 2.0 models and transfers one configuration across benchmarks and model sizes.

Results

70.0%

AIME 2025 pass@1 with LLaDA2.0-Flash

95.0%

AMC 2023 pass@1 with both Flash and Mini

83.3%

AIME 2025 pass@8 candidate upper bound

Steps versus accuracy comparison.
Steps versus accuracy across six benchmarks.
Eight-attempt comparison between Baseline and ED.
Baseline and ED under eight test-time attempts.

BibTeX

@article{sun2026escaping,
  title={Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs},
  author={Sun, Zhenhong and Zhao, Hanqing and Bian, Yatao and Tu, Rong-Cheng and Xie, Liuyue and Zhang, Xu and Wang, Jue and Modolo, Davide and Dong, Daoyi and Tao, Dacheng},
  journal={arXiv preprint},
  year={2026}
}