Physics-based skeletal motion generation by video diffusion distillation
AssigneeNVIDIA Corporation
InventorsXuan Li, Donglai Xiang, Qianli Ma, Tsung-Yi Lin, Ming-Yu Liu, Yongxin Chen
Life of this patent
Estimating the term…
Loading patent…
AssigneeNVIDIA Corporation
InventorsXuan Li, Donglai Xiang, Qianli Ma, Tsung-Yi Lin, Ming-Yu Liu, Yongxin Chen
Estimating the term…
Apparatuses, systems, and techniques for synthesizing motion for three-dimensional (3D) assets. In at least one embodiment, a static 3D asset is obtained based on textual input. The static 3D asset is represented by a 3D spatial representation and an articulated skeleton embedded in the 3D spatial representation. The motion of the articulated skeleton is linked to the deformation of the 3D spatial representation. A sequence of skeleton configurations are generated based on the articulated skeleton, corresponding to a plurality of time steps. A corresponding 3D spatial representation configuration is generated for each skeleton configuration. Based on the sequence of 3D spatial representation configurations, a sequence of image frames are generated to provide a video. A video model evaluates the difference between the video and the textual input to provide a video evaluation loss, which is backpropagated to update the sequence of skeleton configurations.
A computer-implemented method for motion synthesis, comprising:
A system comprising:
A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to:
Dependencies are inferred from the claim text.