Generating synthetic driving scenes using diffusion models

A method comprising:

• receiving, by a scene generator, data representing a driving environment for a synthetic driving scene to be generated;
• receiving, by the scene generator, agent data representing an attribute of a synthetic agent to be generated in the synthetic driving scene;
• determining, based at least in part on the agent data, a first agent token of an agent token type;
• receiving, by the scene generator, desired scene density data for the synthetic driving scene to be generated;
• determining, based at least in part on the desired scene density data, a second density token of a scene density token type;
• and
• providing the first agent token and the second density token as input to a scene diffusion model, and executing the scene diffusion model to generate the synthetic driving scene by iteratively applying a de-noising algorithm to a noise sample, wherein execution of the scene diffusion model is based at least in part on the driving environment, the first agent token, and the second density token. synthetic

One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:

• receiving, by a scene generator, data representing a driving environment for a synthetic driving scene to be generated;
• receiving, by the scene generator, agent data representing an attribute of a synthetic agent to be generated in the synthetic driving scene;
• determining, based at least in part on the agent data, a first agent token of an agent token type;
• receiving, by the scene generator, desired scene density data for the synthetic driving scene to be generated;
• determining, based at least in part on the desired scene density data, a second density token of a scene density token type;
• and
• providing the first agent token and the second density token as input to a scene diffusion model, and executing the scene diffusion model configured to generate the synthetic driving scene by iteratively applying a de-noising algorithm to a noise sample, wherein execution of the scene diffusion model is based at least in part on the driving environment, the first agent token, and the second density token.