Method for automatically animating lip synchronization and facial expression of animated characters
OwnerMaury Rosenfeldinventor
Life of this patent
Estimating the term…
Abstract
A method for controlling and automatically animating lip synchronization and facial expressions of three dimensional animated characters using weighted morph targets and time aligned phonetic transcriptions of recorded text. The method utilizes a set of rules that determine the systems output comprising a stream of morph weight sets when a sequence of timed phonemes and/or other timed data is encountered. Other data, such as timed emotional state data or emotemes such as “surprise, “disgust, “embarrassment”, “timid smile”, or the like, may be inputted to affect the output stream of morph weight sets, or create additional streams.
Claims36
A method for automatically animating lip synchronization and facial expression of three-dimensional characters comprising:
- obtaining a first set of rules that defines a morph weight set stream as a function of phoneme sequence and times associated with said phoneme sequence;
- obtaining a plurality of sub-sequences of timed phonemes corresponding to a desired audio sequence for said three-dimensional characters;
- generating an output morph weight set stream by applying said first set of rules to each sub-sequence of said plurality of sub-sequences of timed phonemes;
- and
- applying said output morph weight set stream to an input sequence of animated characters to generate an output sequence of animated characters with lip and facial expression synchronized to said audio sequence.
An apparatus for automatically animating lip synchronization and facial expression of three-dimensional characters comprising:
- a computer system;
- computer code in said computer system, said computer code comprising:
- a method for obtaining a first set of rules that defines a morph weight set stream as a function of phoneme sequence and times associated with said phoneme sequence;
- a method for obtaining a plurality of sub-sequences of timed phonemes corresponding to a desired audio sequence for said three-dimensional characters;
- a method for generating an output morph weight set stream by applying said first set of rules to each sub-sequence of said plurality of subsequences of timed phonemes;
- a method for applying said output morph weight set stream to an input sequence of animated characters to generate an output sequence of animated characters with lip and facial expression synchronized to said audio sequence.
Dependencies are inferred from the claim text.