# US 2026/0065523 A1 · Sampler for a masked diffusion model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065523 A1
- Document ID: US20260065523A1
- Status: Published application
- Kind: A1
- Application number: 19/294,119
- Filed: 2025-08-07
- Published: 2026-03-05
- Earliest claimed: 2024-08-27
- Assignees: NVIDIA Corporation
- Applicants: NVIDIA Corporation
- Inventors: Qinsheng Zhang; Kaiwen Zheng; Ming-Yu Liu; Yongxin Chen; Hanzi Mao
- Web page: https://patentgrep.com/patent/US20260065523A1

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06T 2210/52: Parallel processing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Masked diffusion models (MDMs), a variant of discrete diffusion formulations, generally use a gradual unmasking process that can generate tokens in any order. These MDMs are useful to generate discrete data, such as text, images, and other sequential data. However, the sampling of MDMs, which is performed in continuous time, traditionally requires that each sampling step make a forward pass through the network even though a single sampling step may result in no changes to any token in the sequence. The present disclosure provides a first hitting sampler for an MDM which, for at least one or more sampling steps, can more efficiently make predictions for unmasking tokens in an input sequence.

## Claims

### Claim 1

A method, comprising:
at a device:
unmasking one or more mask tokens included in an input sequence over a plurality of sampling steps, by a masked diffusion model, to generate an unmasked sequence, wherein a prediction made during at least one sampling step of the plurality of sampling steps is one of:
estimated using linear extrapolation from two or more prior predictions made during respective prior sampling steps of the plurality of sampling steps, or
computed from a current decoding result refined from a prior prediction made during a prior sampling step of the plurality of sampling steps; and
outputting the unmasked sequence.

### Claim 2 (depends on 1)

The method of claim 1, wherein the input sequence is an encoding of an image having one or more masked regions.

### Claim 3 (depends on 2)

The method of claim 2, wherein the unmasked sequence is a complete image.

### Claim 4 (depends on 1)

The method of claim 1, wherein the input sequence is an encoding of a text having one or more masked portions.

### Claim 5 (depends on 4)

The method of claim 4, wherein the unmasked sequence is a complete text.

### Claim 6 (depends on 1)

The method of claim 1, wherein the mask tokens are noisy tokens in the input sequence.

### Claim 7 (depends on 1)

The method of claim 1, wherein the input sequence includes a plurality of mask tokens.

### Claim 8 (depends on 7)

The method of claim 7, wherein the unmasking of at least two mask tokens in the plurality of mask tokens is performed in parallel.

### Claim 9 (depends on 1)

The method of claim 1, wherein the unmasking includes a token-by-token sampling process.

### Claim 10 (depends on 9)

The method of claim 9, wherein at least one mask token is unmasked during each sampling step of the plurality of sampling steps.

### Claim 11 (depends on 1)

The method of claim 1, wherein the prediction made during the at least one sampling step of the plurality of sampling steps is estimated using the linear extrapolation from the two or more prior predictions made during the respective prior sampling steps of the plurality of sampling steps.

### Claim 12 (depends on 11)

The method of claim 11, wherein Lagrange polynomials are used to interpolate the two or more prior predictions along a time axis to estimate the prediction at a current sampling step.

### Claim 13 (depends on 11)

The method of claim 11, wherein the two or more prior predictions include two of the most recent predictions made by the masked diffusion model.

### Claim 14 (depends on 1)

The method of claim 1, wherein the prediction made during the at least one sampling step of the plurality of sampling steps is computed from the current decoding result that has been refined from the prior prediction made during the prior sampling step of the plurality of sampling steps.

### Claim 15 (depends on 14)

The method of claim 14, wherein the current decoding result that has been refined is prevented from being fed back into the masked diffusion model for prediction updates.

### Claim 16 (depends on 1)

The method of claim 1, wherein the at least one sampling step of the plurality of sampling steps makes the prediction without processing through the masked diffusion model.

### Claim 17 (depends on 1)

The method of claim 1, wherein when a number of sampling steps in the plurality of sampling steps is less than or equal to a first threshold, then the prediction made during the at least one sampling step of the plurality of sampling steps is estimated using the linear extrapolation.

### Claim 18 (depends on 17)

The method of claim 17, wherein the first threshold is 128.

### Claim 19 (depends on 1)

The method of claim 1, wherein when a number of sampling steps in the plurality of sampling steps is greater than or equal to a second threshold, then the prediction made during the at least one sampling step of the plurality of sampling steps is computed from the current decoding result.

### Claim 20 (depends on 19)

The method of claim 19, wherein the second threshold is 256.

### Claim 21

A system, comprising:
a non-transitory memory comprising instructions; and
one or more processors in communication with the non-transitory memory, wherein the one or more processors execute the instructions to:
unmask one or more mask tokens included in an input sequence over a plurality of sampling steps, by a masked diffusion model, to generate an unmasked sequence, wherein a prediction made during at least one sampling step of the plurality of sampling steps is one of:
estimated using linear extrapolation from two or more prior predictions made during respective prior sampling steps of the plurality of sampling steps, or
computed from a current decoding result refined from a prior prediction made during a prior sampling step of the plurality of sampling steps; and
output the unmasked sequence.

### Claim 22 (depends on 21)

The system of claim 21, wherein the input sequence includes a plurality of mask tokens, and wherein the unmasking of at least two mask tokens in the plurality of mask tokens is performed in parallel.

### Claim 23 (depends on 1)

The method of claim 1, wherein the unmasking includes a token-by-token sampling process, and wherein at least one mask token is unmasked during each sampling step of the plurality of sampling steps.

### Claim 24

A non-transitory computer-readable media storing computer instructions which when executed by one or more processors of a device cause the device to:
unmask one or more mask tokens included in an input sequence over a plurality of sampling steps, by a masked diffusion model, to generate an unmasked sequence, wherein a prediction made during at least one sampling step of the plurality of sampling steps is one of:
estimated using linear extrapolation from two or more prior predictions made during respective prior sampling steps of the plurality of sampling steps, or
computed from a current decoding result refined from a prior prediction made during a prior sampling step of the plurality of sampling steps; and
output the unmasked sequence.

### Claim 25 (depends on 24)

The non-transitory computer-readable media of claim 24, wherein the input sequence includes a plurality of mask tokens, and wherein the unmasking of at least two mask tokens in the plurality of mask tokens is performed in parallel.

### Claim 26 (depends on 24)

The non-transitory computer-readable media of claim 24, wherein the unmasking includes a token-by-token sampling process, and wherein at least one mask token is unmasked during each sampling step of the plurality of sampling steps.
