Music reactive animation of human characters
AssigneeSnap Inc.
InventorsGurunandan Krishnan Gorumkonda, Hsin-Ying Lee, Jie Xu
Life of this patent
Estimating the term…
Abstract
Example methods for generating an animated character in dance poses to music may include generating, by at least one processor, a music input signal based on an acoustic signal associated with the music, and receiving, by the at least one processor, a model output signal from an encoding neural network. A current generated pose data is generated using a decoding neural network, the current generated pose data being based on previous generated pose data of a previous generated pose, the music input signal, and the model output signal. An animated character is generated based on a current generated pose data; and the animated character caused to be displayed by a display device.
Claims20
A method for generating an animated character in dance poses to music, the method comprising:
- generating, by at least one processor, a music input signal based on an acoustic signal associated with the music;
- receiving, by the at least one processor, a model output signal from an encoding neural network;
- generating, by the at least one processor, current generated pose data using a decoding neural network, the current generated pose data being based on previous generated pose data of a previous generated pose, the music input signal, and the model output signal;
- generating, by the at least one processor, an animated character based on a current generated pose data;
- and
- causing, by the at least one processor, the animated character to be displayed by a display device.
A computing apparatus comprising:
- at least one processor;
- and
- a memory storing instructions that, when executed by the at least one processor, configure the apparatus to:
- generate a music input signal based on an associated music acoustic signal;
- receive a model output signal from an encoding neural network;
- generate a current generated pose data using a decoding neural network, the current generated pose data being based on previous generated pose data of a previous generated pose, the music input signal, and the model output signal;
- generate an animated character based on the current generated pose data;
- and
- cause the animated character to be displayed by a display device.
A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
- generate, by at least one processor, a music input signal based on an associated music acoustic;
- receive, by at least one processor, a model output signal from an encoding neural network;
- generate, by at least one processor, current generated pose data using a decoding neural network, the current generated pose data being based on a previous generated pose of a previous generated pose, the music input signal, and the model output signal;
- generate, by at least one processor, an animated character based on the current generated pose data;
- and
- cause, by at least one processor, the animated character to be displayed by a display device.
Dependencies are inferred from the claim text.