Denoising Models Develop Human-Like Perceptual Illusion Representations Across Architectures

Gautam Ranka · Paras Chopra

arXiv:2607.17138 · 19 Jul 2026 · Preprint

A short walkthrough of the paper's central finding: the illusion shows up inside the model, grows strongest partway through, and fades away again before it ever reaches the final image.

Static brightness-contrast demonstration: two identical grey patches, one on a dark surround, one on a light surround

Both grey squares are exactly the same colour, pixel for pixel. The one on the dark background looks lighter, purely because of what surrounds it: a brightness illusion. This is the kind of manipulation the paper traces inside a denoising model.

|d| = 0.663peak illusion sensitivity

43.8%signal drop on ablation

shift_frac ≈ zeropixel shift at the output

ρ = 0.78match to human perception

2.5×objective vs. architecture

N = 35illusion images probed

Abstract

Deep neural networks trained on natural images are shown to produce outputs consistent with human observers for brightness illusions. While this phenomenon has been documented across architectures, all evidence, to date, is measured at the output level: restored pixels, decoded trajectories, or classification decisions. Whether these models actually represent illusions internally, and if so where and how, remains unknown. We show that denoising models develop illusion-sensitive representations at specific internal layers, across varied architectures. Specifically, we identify the layers and channels that discriminate illusory from physically matched control regions. We show that the denoising objective is a more important driver of the effect than the architecture. On domain-appropriate stimuli, these activations track a validated psychophysical model of human brightness perception (FLODOG; Spearman ρ ≥ 0.70) and scale monotonically with parametric illusion strength. Leveraging these findings, we provide causal evidence via channel ablation showing that illusion-sensitive channels specifically and substantially affect the internal signal. Yet injecting these representations into the generation pipeline produces no measurable pixel shift across all tested architectures; we term such representations perceptual phantoms: active in internal processing but invisible to any output-based evaluation. While related internal-output dissociations have been characterized in language models, this is the first such characterization for perceptual representations in denoising vision models.

Three claims

Denoising models encode illusions internally, regardless of architecture. Every denoising model tested, including both U-Nets and diffusion transformers, develops illusion-sensitive representations at specific layers. Similar discriminative models show consistently weaker effects. The denoising objective, not architecture, appears to be what matters.

The encoding is causally involved in internal processing. Targeted channel ablation demonstrates that illusion-sensitive channels play a causal role in shaping the internal representations. Specific channels dominate the effect, and the signal concentrates at the bottleneck rather than in output-bound pathways.

Illusion encoding causally attenuates en route to the output. The encoding propagates downstream, but with progressively diminishing output coupling, to finally reach a point where illusion-specific channels disturb pixel reconstruction less than matched random channels: a perceptual phantom.

Methods

9 models (6 denoising models across 3 architecture families, plus 3 discriminative baselines) are probed with identical stimuli and a single-step forward-diffusion pass.

Model	Role	Size / resolution
Pixel-space DDPMs
`google/ddpm-ema-church-256`	primary model	113.7M, 256×256
`google/ddpm-ema-bedroom-256`	LSUN Bedrooms	-
`google/ddpm-ema-celebahq-256`	CelebA-HQ faces	-
Latent diffusion U-Nets
`CompVis/ldm-celebahq-256`	LDM, CelebA-HQ	-
`runwayml/stable-diffusion-v1-5`	`SD 1.5`	latent 64×64
Diffusion transformer
`DiT-XL/2`	ImageNet class-conditioned, no U-Net structure	28 blocks, 675M
Discriminative baselines
`ResNet-50`	ImageNet	-
`VGG-19`	ImageNet	-
`ViT-B/16`	ImageNet	-

All models frozen throughout, single forward pass per image per timestep. Parameter counts for the remaining six models are given in the paper's appendix and are not reproduced here.

Full methodological detail

Stimuli

GVIL: 35 base colour illusion images, 2 illusion regions each, with matched real-counterpart (X_r) images in which the illusory context is removed. (The original GVIL set has 72 images; the larger set is excluded here because it includes flip augmentations.) Plus Pyllusion parametric Ebbinghaus and Ponzo stimuli at 10 strength levels, 10 images per level, for dose-response analysis.

Probing

x_{t} = \sqrt{{\bar{α}}_{t}} x_{0} + \sqrt{1 - {\bar{α}}_{t}} ε, ε \sim N (0, I)

A single-step forward diffusion pass; activations captured via forward hooks. Primary claims are based on t ∈ {50, 150}; t > 700 is excluded for a heavy-noise confound.

Effect sizes and a-priori layers

Cohen's d, Hedge's g, and Glass's Δ, with four effect-size bands defined by the boundaries 0.2 / 0.5 / 0.8 (negligible / small / medium / large). Three layers were fixed as a-priori targets before observational analysis, so all subsequent analysis at them is confirmatory: mid_attn_0 (bottleneck attention), mid_resnet_0 (bottleneck ResNet), down_5_resnet_1 (deepest encoder layer).

Controls

Pixel-Shuffle within illusion_a, dissociating gestalt from feature processing.
Real-Counterpart comparison (Δ_ill/Δ_real), separating the illusion effect from image-region statistics.
Random Initialization, separating training from architecture.
Random-Image baselines with matched mask geometry, tested via d_excess.
Multiple-hypothesis correction via 10,000-permutation tests with FDR.

Where the illusion lives

The signal is not spread evenly across the network. It concentrates sharply at the bottleneck, then fades to almost nothing by the time the model produces its output.

As an image moves through the network, the model's internal response to the illusion rises, peaks sharply at the bottleneck between the encoder and decoder, then fades again well before the output. Hover or tap a stage to see what happens there.

It tracks human perception

FLODOG is a validated model of human brightness perception. On domain-appropriate luminance stimuli, U-Net activations align with it closely, and the correlation scales monotonically with illusion strength.

Each bar measures how closely the model's internal activity tracks a validated model of human brightness perception. The closer a bar comes to filling its track, the more the model's response resembles what a human visual system does with the same stimulus.

Causal, and a phantom

Ablating illusion-sensitive channels reduces the internal signal substantially, but disturbs the final output less than ablating random channels, and injecting the representations produces no measurable pixel shift at all.

shift_frac per released row

Each dot is one experiment: the illusion-sensitive activity was transplanted into a clean image, and the resulting change in the output pixels was measured. Almost every dot sits right on the centre line, meaning the transplant left the final image essentially untouched, no matter which model was tested.

The objective, not the architecture

Every denoising model clears the medium-effect threshold; every discriminative baseline falls below it. Holding the architecture fixed and changing only the training objective reproduces the same gap.

Each bar is one model's strongest response to the illusion. Denoising models, in blue, consistently clear the line marking a meaningful effect; discriminative classifiers, in orange, trained only to label images rather than reconstruct them, mostly do not.

Diffusion transformers process an image as a sequence of blocks rather than an encoder and decoder. Traced block by block, the same pattern shows up again: a flat response through most of the network, then a sharp spike near the end.

Limitations

Small N. N=35 is the primary limitation. While bootstrap confidence intervals confirm the headline claims, some secondary results remain underpowered. Synthetic and ablated images were avoided to prevent inflated results, such as the 89% increase in measured ablation reduction that using them would have produced.
Psychophysical alignment limited to luminance. FLODOG ρ applies only to SBC and Hermann-grid stimuli. Developing or adapting chromatic models would enable graded psychophysical validation on the full GVIL stimulus set.
Training-vs-architecture is complex. Trained models consistently exceed the random-seed median, but some random seeds match trained performance in DiT and in dose-response (ρ ≈ 0.95, sign-inconsistent). The denoising objective is the common thread, but individual-seed variability means architecture and training cannot be fully separated.
Phantom characterization is empirical, not exhaustive. The phantom property is established for one specific class of representations. Not all internal representations in denoising models are claimed to be phantoms, nor is the phenomenon claimed unique to perception. Characterizing which representations are read-only versus read-write remains open.

Citation

@article{ranka2026denoising,
  title   = {Denoising Models Develop Human-Like Perceptual Illusion Representations Across Architectures},
  author  = {Ranka, Gautam and Chopra, Paras},
  journal = {arXiv preprint arXiv:2607.17138},
  year    = {2026}
}