POPE
300–400 examples per measurement pool; 300 per backbone in the utilization panel. Is a named object present in the image? Easy yes/no regime; tests invented objects and early answer commitment.
Source and method ↗Research companion · methods, evidence and open materials
Do dVLMs Use Diffusion?
Follow how a diffusion vision-language model reaches an answer—and what its remaining decoding passes change.
01 · Overview
The question and main answer
Start with what was studied, what the evidence says, and the scope of the comparison.
Main finding
The paper's abstract
02 · Example
A concrete example
See the phenomenon before reading the method or comparing aggregate results.
One question at a time
2.1 Follow one recorded decode
In plain wordsPick a real question and press play. You see what the model would answer if you stopped it at each pass, and how unsure it is about the answer word. The answer word settles within a few passes; the uncertainty falls much later.
Pick a question and press play. The box shows what the model would output if it stopped at that pass; the highlighted word is the answer slot. The chart is the entropy at the answer slot (log scale). The answer stops changing long before the entropy collapses, and the text keeps changing after that.
Kind
03 · Method
How the experiment works
The study reads the answer, confidence and full text after successive calls to the same denoising model. The sampler loop and the neural network have different roles.
See it in 3D
3.1 Inside the model, around the sampler
Separate the neural network from the algorithm that calls it repeatedly. Follow image features, the answer canvas and per-position logits; then inspect the experimental instruments around the decoding loop.
How to use it. Choose a numbered step to see what changes and what to look at. Select a stage name to inspect it. Drag to orbit, use + and − to zoom, or play the story. The full text works without 3D.
SYSTEM ARCHITECTURE · 1 / 9
14 components · 17 connectionsEncode the image
LLaDA-V: SigLIP2 → LLaDA-8B. Single-decode measurements at T=32.
DataFeedbackInterventionMeasurementIllustrative geometry · no activation values encoded
Interactive 3D loads when this map comes into view. Component inputs, outputs and connections are available below.
Drag to orbit · select a component to inspect
The connected map and all explanations work without WebGL.
STEP 1 OF 9
Encode the image
Inspect the visual pathway of the selected backbone. The detailed LLaDA-V view separates the image encoder from its projector.
Look at the highlighted components.
Inspect the measured evidence ↓Explore a component in detail
How to read the diagram and its limits
Horizontal boards separate setup, computation and evaluation. Exploded layers lifts these boards apart; it does not add neural layers. Solid blue paths carry data; amber paths loop back. Purple dashed paths are interventions; green dashed paths are measurements. Arrows show direction. Moving packets illustrate flow and do not encode measured speed, confidence or activation. Use the inspector to read every connection. Layer counts and dimensions are stated only where supported by the selected model.
The overview is a system schematic. Exact block details shown for LLaDA-V come from its released architecture metadata; other backbones have different visual interfaces and samplers. The 32 model layers are not the 32 denoising passes. Arrows show information paths, not measured activation strength. Early-stop certification bounds answer changes relative to the full schedule, not truth.
Architecture
3.2 Where the measurements are taken
In plain wordsThe picture is turned into tokens once. Then the language model fills in a fully hidden answer over several passes: each pass it guesses every hidden word, keeps the surest ones and hides the rest again. The paper reads the model at every pass, at no extra cost.
The image is encoded once. The language model then fills in a fully masked answer over T passes. At each pass it predicts a word for every masked position, keeps the most confident one and masks the rest again. Press play to send one real question through the loop: the box under the diagram shows what the model would answer if it stopped at that pass.
pass – of 32
The account in one picture
3.3 The measurements and research questions
In plain wordsA diffusion VLM writes its whole answer at once and then refines it over several passes. We read what it would answer, and how sure it is, after every pass. The answer settles first; the confidence comes much later; the rest of the text settles last.
A masked-diffusion VLM starts from a fully masked answer and fills it in over T forward passes. At every pass we read the answer the model would give if stopped there, the entropy at the answer position, and the full text. On LLaDA-V the three settle at very different times.
One decode, three clocks · LLaDA-V, POPE, T = 32 · means over examples
The paper asks four questions
Figure of the paper · the architecture, and where each measurement is taken
More method detail
The full written explanation, study inventory and original replays are available here.
Written walkthrough
Start here
3.4 The whole study in plain words
An everyday pictureThink of a student who writes the answer on the first line of the exam and spends the rest of the time writing an explanation underneath. By the end the student feels very sure, but only because the explanation agrees with the answer already written, not because they looked at the picture again. The models we study behave the same way.
New to the words? The glossary explains every term on this page in one or two sentences.
Study design and test conditions
Scope
3.5 How much we tested
A claim that diffusion VLMs barely use their schedule is only as strong as the search behind it. Here is the size of that search, as the paper reports it.
More about the study design and test conditions
The seven backbones · three language-model families, five kinds of image encoder
The benchmarks
Complete models and datasets
Scope
3.6 The complete study inventory
Here is what was actually tested. The model roles and sample sizes belong to specific experiments; they are not interchangeable coverage claims.
Explore all 7 models and their roles
7 models shown
LLaDA-V
Primary instrumented modelLLaDA-8B SigLIP2 Single-decode measurements at T=32.
Source and method ↗LaViDa-LLaDA
Generalization backboneLLaDA-8B SigLIP Single-decode measurements at T=32.
Source and method ↗MMaDA-8B
Detailed cross-lineage comparisonMMaDA-8B MAGVIT-v2 tokens Single-decode measurements at T=32.
Source and method ↗Dream-VL
Generalization backboneDream-7B Qwen2-VL ViT Single-decode measurements at T=32.
Source and method ↗LaViDa-Dream
Generalization backboneDream-7B SigLIP GQA was not run for this backbone.
Source and method ↗Dimple
Generalization backboneDream-7B Qwen2-VL ViT Single-decode measurements at T=32.
Source and method ↗Nemotron-D
Generalization backboneNemotron-Diffusion-8B Pixtral Single-decode measurements at T=32.
Source and method ↗Explore datasets, sample sizes and what each test measures
GQA (two-hop / relational subset)
300 examples per backbone cell. Questions requiring relations between objects in a real photograph. Hard regime with more room for improvement; six backbone cells.
Source and method ↗POPE fresh adversarial and random splits
300 distinct images per split, one question per image. Checks that results are not specific to the original question pool.
Source and method ↗POPE adversarial + Gaussian image noise (σ=0.2)
Matched questions from the adversarial split. Tests whether the predicted effect persists when visual evidence is degraded.
Source and method ↗Rendered disc evidence-strength sweep
200 trials. The intended difficulty manipulation failed: accuracy was 100% at every radius. Only the pooled decision–confidence gap is interpretable.
Source and method ↗A model count measures breadth. It does not mean every causal claim holds in every cell. The page must say which mechanism was present and which outcome was tested.
Coverage is explicit. Seven POPE cells and six GQA cells make 13—not 14—backbone × benchmark combinations. LaViDa-Dream / GQA was not run.
| Backbone | POPE | GQA |
|---|---|---|
| LLaDA-V | Tested | Tested |
| LaViDa-LLaDA | Tested | Tested |
| MMaDA-8B | Tested | Tested |
| Dream-VL | Tested | Tested |
| LaViDa-Dream | Tested | Not run |
| Dimple | Tested | Tested |
| Nemotron-D | Tested | Tested |
Explore the original experiment replay and measured trajectories
See it in 3D
The model and the whole argument, in 3D
How to use itThis is LLaDA-V, the main model of the study, built as a building: the picture and the question enter on the left, an image encoder and a language model process them, and the answer is written into a tower whose floors are the denoising passes, one floor per pass. The eleven steps follow the paper: the model, then its four questions, then its conclusion. The caption above the picture says what each step shows and where to look, and the colours button explains the colours. Press play or click the numbers; drag to turn the model, use + and − to zoom, and hover any part for its name.
Loading the 3D view…![Architecture of a masked-diffusion VLM (the paper's Figure 1)]()
drag to turn
What the colours mean
gold · a hedge (“There, there is a …”) in the answer slot, or an answer frozen too early
green · the answer word is “yes” or “no”, decided
blue · the decision clock, the image encoder and the tokens
orange · the confidence clock: uncertainty at the answer word
red · a wrong final answer, or a result below chance
glass floors · one floor per denoising pass
04 · Results
What the experiments found
Read the comparisons in order. Each result keeps its controls, counts and source evidence together.
- How much of the schedule changes the answer?
- When does confidence arrive?
- What does token commitment change?
- Does confidence transfer across schedules?
- Do endpoint repairs help?
- Can sampling stop earlier?
Q1
4.1 How much of the schedule changes the answer?
In plain wordsHow much of the 32-pass schedule can still change the answer? Very little. On POPE the answer is final after to passes on every backbone, and most answers never change after the first pass. Harder GQA questions use more of the schedule.
Iteration utilisation U is the last pass at which the emitted answer changes, divided by the number of passes: the share of the schedule that can still alter the answer. It needs no extra computation.
Iteration utilisation · share of the 32-pass schedule that can still change the answer · Table
POPEGQA (harder)95% bootstrap interval
BenchmarkHighlight
When the answer is final · cumulative share of answers that never change again after pass k
Q2
4.2 When does confidence arrive?
In plain wordsThe model becomes sure of its answer only later, and a simple test shows why. Give it the text it has already written and it becomes certain, whether the image is real or blank; scramble the same words and it does not. Its confidence measures how well the answer fits its own text, not the picture.
Two timescales · as the schedule lengthens · LLaDA-V, POPE, n = matched
answer decided (τdec)confident, entropy < 0.1 (τconf)
What the confidence responds to · answer-slot entropy given the model's own text · n = · log scale
The substitution test, step by step · one extra forward pass after decoding, with one input changed
The substitution test · every one of examples · answer-slot entropy, log scale
after the first passthe model's own final text suppliedthe same tokens, scrambled
Q3
4.3 What does token commitment change?
In plain wordsLonger schedules are more accurate, but not because the model thinks longer. A short schedule writes many words at once, and its first round can lock a non-answer such as “There, there is a …” into the answer slot. Keeping the answer out of that first round, the answer-deferred round, fixes it.
Short schedules commit many tokens per pass. When the first pass puts a hedge template at the answer slot ("There, there is a …"), a coarse first round freezes it in place. The answer-deferred round (ADR) keeps the answer span out of the first commit round; nothing else changes.
Why a short schedule freezes a hedge · schematic of the commit rounds · the numbers are measured on POPE examples
The freeze law · POPE accuracy by the state of the answer slot after the first pass
decided at pass 1 (n = )hedge at pass 1 (n = )hedge examples that were not frozen
One deferred round · POPE accuracy at the same forward passes
stock samplerwith ADRfull 32-pass schedule
The hedge proposition, tested · top-1 probability at the answer slot after the first pass · LLaDA-V, POPE
hedge at pass 1 (n = )decided at pass 1 (n = )the window ⅓ ≤ p < ½ the proposition predicts
Reliability
4.4 Does confidence transfer across schedules?
In plain wordsBecause the confidence tracks the model’s own text, it gets worse at spotting wrong answers as the schedule gets longer, even while accuracy rises. Reading the same uncertainty early, before the text exists, works much better and costs nothing.
If the confidence tracks the text, a longer schedule that writes more text should make it a worse error detector, even as accuracy rises. Reading the same entropy at a fixed early pass avoids that.
Pool
Schedule fragility · AUROC for ranking errors · LLaDA-V, POPE
endpoint entropyearly read (pass 3)full trajectory
Cell
Read once, early · AUROC of the answer-slot entropy read at pass k
read at pass k, with its intervalendpointfull trajectory
An exhaustion result
4.5 Do endpoint repairs help?
In plain wordsWe tried several ways to repair the final confidence, all fixed before looking at the data. None comes close to the simple early signal.
The interventions were fixed before seeing the data. None approaches the free trajectory statistic as an error detector; on GQA three rank errors below chance.
POPE · error-detection AUROC · n =
GQA · error-detection AUROC · n =
endpoint repairsupervised 7-signal head (cross-validated, no interval)free trajectory statistic95% bootstrap interval
Compute
4.6 Can sampling stop earlier?
In plain wordsThe idle passes can be skipped safely. A statistical certificate decides when to stop and keeps the chance that stopping changes the answer below a chosen budget. Running the same procedure with a random score shows how much of the saving the score itself earns.
The exit threshold is certified with distribution-free bounds at a flip-risk budget α. The same procedure run on a random score shows how much of the saving any score would get; on POPE flips are rare and the certificate is largely vacuous, on GQA it is not.
CellShow
Share of the computation saved · at each flip-risk budget
trajectory scorerandom score (control)
The exit, animated ·
05 · Reliability
Reliability and limits
Check the alternative explanations, generalization tests and boundaries of the claim.
- How broadly does the result hold?
- Controls and alternative explanations
- What the study does not establish
Q4
5.1 How broadly does the result hold?
In plain wordsThe same pattern holds on seven models from three families, on fresh questions and under image noise. Asking a model to explain itself creates the pattern exactly as the account predicts.
The account on seven backbones · paired AUROC differences with 95% intervals · Table
early read Δ (pass 3 − endpoint)fragility Δ (endpoint at T = 4 minus at T = 32)
ADR on fresh pools and under noise · LLaDA-V, accuracy gain at T = 4 · Table
Prompting a model to explain · ADR gain at T = 4 when asked to write a rationale
Trust and boundaries
5.2 Controls and alternative explanations
A result is more useful when we can see the alternative explanations it survived—and where it stops holding.
Change the question pool and damage the image
Repeat the answer-deferred first round on fresh adversarial and random POPE pools, then add Gaussian noise to the adversarial images. Accuracy gain at four passes: +8.7 points [4.3, 13.0] adversarial; +12.7 [8.3, 17.0] random; +14.0 [8.7, 19.3] with noise.
On GQA the gain is +2.0 points [−2.7, 6.7], which is not statistically clear.
Source and method ↗Separate the text from the picture
Hide the answer slot again, give the model its own final text, scramble those same tokens, or replace the real image with a blank canvas. The 250-example LLaDA-V substitution test loses about 99.8% of answer-slot entropy with its own text. In a separate 120-example blank-image control, the image difference is +0.00009 nats [−0.00011, +0.00031]; scrambling the text raises entropy by about 2.1 nats.
This measures answer-slot confidence under this supplied context; it does not show that the model ignores the image for all computations.
Source and method ↗Make the proposed cause appear in another model
Ask bare-answer models to explain, then verify whether they actually produce surrounding text before interpreting the intervention. Only Dimple and Nemotron-D followed the rationale instruction. Dimple then develops the frozen first round; deferring it restores +15.7 accuracy points [11.3, 20.0] on POPE.
Dream-VL and LaViDa-LLaDA ignored the instruction. Those runs are replications, not successful tests of this manipulation.
Source and method ↗Check a certificate against a random score
Run the identical certified exit procedure with the measured trajectory score and a random score in each model, benchmark and risk-budget cell. At a 5% flip-risk budget on LLaDA-V/GQA, the trajectory score saves 54.91% of forward-pass compute and the random score saves 0%.
The control is strong only at specified cells. On POPE, random already saves 65.6% versus 71.4% for LLaDA-V. On GQA, random also saves compute at more permissive budgets (43.75% at 10%, 68.75% at 20%).
Source and method ↗Show where the account predicts no benefit
Compare across seven backbones, then separate cases with answer-bearing surrounding text from cases with an intact bare-answer endpoint. Early settlement appears in all 13 utilization cells. The early-confidence read helps where endpoint confidence is degraded; the deferred round does nothing when the first round freezes no hedge.
The early read is not universally helpful: Dream-VL/POPE loses about 0.14 AUROC. Intervals are over examples, not repeated generation seeds.
Source and method ↗Limits
5.3 What the study does not establish
In plain wordsWhat our results do not show, in the paper’s own words.
Additional scope notes from the companion guide
Read the claim within these limits
- The confidence account concerns answer slots read after self-committed context. LLaDA-V and Nemotron-D naturally provide that text; Dimple provides it after a successful prompt intervention.
- These cells use one decode per example. Bootstrap intervals describe example-level variation; they do not measure seed-to-seed generation variability.
- Dream-style backbones use Dream’s fixed-step confidence sampler. For Dimple this replaces the native pipeline, so the result describes that instrumented sampler.
- The exit certificate bounds flips relative to the full 32-pass answer. It is a marginal risk guarantee, not a probability that any individual answer is correct.
- MMaDA is measured at the answer position only; multi-word GQA answers are compared using their first token. The entropy threshold was calibrated on LLaDA-V and does not transfer unchanged.
- Forward-pass savings and wall-clock speed are reported separately. Skipping passes can save computation without the same percentage change on every hardware stack.
06 · Materials
Research materials
Open the material you need: paper figures and tables, code, result files and reproduction instructions.
Paper tables
Results
6.1 Every table of the paper and its supplement
In plain wordsAll tables of the paper and its supplement, each with the paper’s own note on how to read it where there is one. Type in the box to find one.
Paper figures
Visualize
6.2 Every figure of the paper and its supplement
In plain wordsAll figures, with their captions. Click one to enlarge it.
Click a figure to enlarge it and read its caption.
Code browser
Code
6.3 All the code: experiments, analyzers, tables and figures
In plain wordsThe scripts that ran the experiments, the ones that analyse their results, and the ones that draw every table and figure.
The data module recomputes every number from the per-example records; the experiment drivers wrote those records on one GPU each.
Raw result files
Result files
6.4 Every number traces to one of these files
In plain wordsThe result files behind the paper’s numbers. Under each file: the scripts that read it.
Reproduction instructions
Reproduce
6.5 Rebuild the tables and figures from the archive
In plain wordsDownload one archive and run these commands to rebuild the paper’s tables and figures.
Download the archive, unzip it and run from its top folder (Python 3.9+ with numpy, scipy, matplotlib and pillow).
python3 papers/cvpr2027_dvlm/make_supp_tables.py # every generated table python3 papers/cvpr2027_dvlm/make_backbone_tables.py # the main-text and backbone tables python3 papers/cvpr2027_dvlm/make_supp_figures.py # the supplementary figures (and the line figures) python3 papers/cvpr2027_dvlm/supp_data.py # recompute the main cache from the per-example records python3 papers/cvpr2027_dvlm/supp_data_backbones.py # recompute the seven-backbone cache (about 1.5 minutes)
Glossary
Words