Sampler for a masked diffusion model
AssigneeNVIDIA Corporation
InventorsQinsheng Zhang, Kaiwen Zheng, Ming-Yu Liu, Yongxin Chen, Hanzi Mao
Life of this patent
Estimating the term…
Loading patent…
AssigneeNVIDIA Corporation
InventorsQinsheng Zhang, Kaiwen Zheng, Ming-Yu Liu, Yongxin Chen, Hanzi Mao
Estimating the term…
Masked diffusion models (MDMs), a variant of discrete diffusion formulations, generally use a gradual unmasking process that can generate tokens in any order. These MDMs are useful to generate discrete data, such as text, images, and other sequential data. However, the sampling of MDMs, which is performed in continuous time, traditionally requires that each sampling step make a forward pass through the network even though a single sampling step may result in no changes to any token in the sequence. The present disclosure provides a first hitting sampler for an MDM which, for at least one or more sampling steps, can more efficiently make predictions for unmasking tokens in an input sequence.
A method, comprising:
A system, comprising:
A non-transitory computer-readable media storing computer instructions which when executed by one or more processors of a device cause the device to:
Dependencies are inferred from the claim text.