Physics-guided motion diffusion model

A method, comprising:

• processing, by a diffusion neural network model, an input motion definition and noisy motion data to produce synthesized motion data for a character for a diffusion timestep of multiple diffusion timesteps, wherein the synthesized motion data comprises joint positions and rotations for the character;
• selectively disabling processing by a physics-based motion neural network model based on a mean motion of a denoising distribution and a noise level of the synthesized motion data;
• converting the joint positions and rotations for the character into joint angles and velocities;
• for each diffusion timestep that a physics-based motion neural network model is not disabled, applying learned parameters, by the physics-based motion neural network model, to adapt the synthesized motion data to physical constraints of an environment on the character;
• converting adapted joint angles and velocities generated by the physics-based motion neural network model into the physics-guided motion data to produce physics-guided motion data that is specific to the character and physically plausible, wherein the physics-guided motion data corresponds to the input motion definition;
• for each specific diffusion timestep that the physics-based motion neural network model is disabled, providing the synthesized motion data as the physics-guided motion data;
• and
• repeating the processing and applying for one or more remaining diffusion timesteps, wherein the physics-guided motion data replaces the noisy motion data for each iteration of the processing and applying.

A method, comprising:

• processing, by a diffusion neural network model, an input motion definition and noisy motion data to produce synthesized motion data for a character for a diffusion timestep of multiple diffusion timesteps;
• selectively disabling processing by a physics-based motion neural network model based on a mean motion of a denoising distribution and a noise level of the synthesized motion data;
• for each diffusion timestep that a physics-based motion neural network model is not disabled, applying learned parameters, by the physics-based motion neural network model, to adapt the synthesized motion data to physical constraints of an environment on the character producing physics-guided motion data that is specific to the character and physically plausible, wherein the physics-guided motion data corresponds to the input motion definition and at least one of the steps of processing and applying is performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle;
• for each specific diffusion timestep that the physics-based motion neural network model is disabled, providing the synthesized motion data as the physics-guided motion data;
• and
• repeating the processing and applying for one or more remaining diffusion timesteps, wherein the physics-guided motion data replaces the noisy motion data for each iteration of the processing and applying.

A system, comprising a processor configured to produce physics-guided motion data by:

• processing, by a diffusion neural network model, an input motion definition and noisy motion data to produce synthesized motion data for a character for a diffusion timestep of multiple diffusion timesteps;
• selectively disabling processing by a physics-based motion neural network model based on a mean motion of a denoising distribution and a noise level of the synthesized motion data, wherein the mean motion corresponds to physically-incorrect motion;
• for each diffusion timestep that a physics-based motion neural network model is not disabled, applying learned parameters, by the physics-based motion neural network model, to adapt the synthesized motion data to physical constraints of an environment on the character producing the physics-guided motion data that is specific to the character and physically plausible, wherein the physics-guided motion data corresponds to the input motion definition;
• for each specific diffusion timestep that the physics-based motion neural network model is disabled, providing the synthesized motion data as the physics-guided motion data;
• and
• repeating the processing and applying for one or more remaining diffusion timesteps, wherein the physics-guided motion data replaces the noisy motion data for each iteration of the processing and applying.

A non-transitory computer-readable media storing computer instructions for producing physics-guided motion data that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• processing, by a diffusion neural network model, an input motion definition and noisy motion data to produce synthesized motion data for a character for a diffusion timestep of multiple diffusion timesteps;
• selectively disabling processing by a physics-based motion neural network model based on a mean motion of a denoising distribution and a noise level of the synthesized motion data, wherein the mean motion corresponds to physically-incorrect motion;
• for each diffusion timestep that a physics-based motion neural network model is not disabled, applying learned parameters, by the physics-based motion neural network model, to adapt the synthesized motion data to physical constraints of an environment on the character producing the physics-guided motion data that is specific to the character and physically plausible, wherein the physics-guided motion data corresponds to the input motion definition;
• for each specific diffusion timestep that the physics-based motion neural network model is disabled, providing the synthesized motion data as the physics-guided motion data;
• and
• repeating the processing and applying for one or more remaining diffusion timesteps, wherein the physics-guided motion data replaces the noisy motion data for each iteration of the processing and applying.

A method, comprising:

• processing, by a diffusion neural network model, an input motion definition and noisy motion data to produce synthesized motion data for a character for a diffusion timestep of multiple diffusion timesteps;
• selectively disabling processing by a physics-based motion neural network model based on a mean motion of a denoising distribution and a noise level of the synthesized motion data;
• for each diffusion timestep that a physics-based motion neural network model is not disabled, applying learned parameters, by the physics-based motion neural network model, to adapt the synthesized motion data to physical constraints of an environment on the character producing physics-guided motion data that is specific to the character and physically plausible, wherein the physics-guided motion data corresponds to the input motion definition;
• for each specific diffusion timestep that the physics-based motion neural network model is disabled, providing the synthesized motion data as the physics-guided motion data, wherein a number of the diffusion timesteps for which the physics-based motion neural network model is not disabled is specified and further comprising selectively disabling the physics-based motion neural network model based on the number;
• and
• repeating the processing and applying for one or more remaining diffusion timesteps, wherein the physics-guided motion data replaces the noisy motion data for each iteration of the processing and applying.

A method, comprising:

• processing, by a diffusion neural network model, an input motion definition and noisy motion data to produce synthesized motion data for a character for a diffusion timestep of multiple diffusion timesteps;
• selectively disabling processing by a physics-based motion neural network model based on a mean motion of a denoising distribution and a noise level of the synthesized motion data;
• for each diffusion timestep that a physics-based motion neural network model is not disabled, applying learned parameters, by the physics-based motion neural network model, to adapt the synthesized motion data to physical constraints of an environment on the character producing physics-guided motion data that is specific to the character and physically plausible, wherein the physics-guided motion data corresponds to the input motion definition;
• for each specific diffusion timestep that the physics-based motion neural network model is disabled, providing the synthesized motion data as the physics-guided motion data;
• and
• repeating the processing and applying for one or more remaining diffusion timesteps. wherein the physics-guided motion data replaces the noisy motion data for each iteration of the processing and applying and the physics-based motion neural network model is selectively disabled for at least one diffusion timestep in an early portion of the multiple diffusion timesteps.