Character animation system

A system for generating an animated image of at least a head of a character which is speaking, the character's face having visible articulation matching words being spoken, the system comprising:

• a first processing module for processing a first stream of marked up text comprising text to be spoken by an animated character and a plurality of mark up instructions representing behavioural features to be implemented in the animated character, so as to convert said first stream of marked up text into a second, modified, stream of marked up text by replacing recognisable ones of said mark up instructions in said first stream with predefined modified mark up instructions, wherein said first processing module includes mark up instruction recognition means comprising comparing means for comparing mark up instructions in said first stream with predetermined mark up instructions stored in a memory means accessible to the system;
• a second processing module comprising randomiser means for processing said second stream of marked up text so as to insert randomly selected mark up instructions into said second stream of marked up text, said randomly selected mark up instruction representing random behavioural features to be implemented in the animated character;
• a third processing module comprising text-to-speech converter means for processing said second stream of marked up text so as to produce:
• an audio signal stream for use in generating an audio signal representing said text being spoken;
• a phoneme stream representing a sequence of phonemes corresponding to successive portions of the audio signal;
• and a mark up instruction stream comprising mark up instructions for use in other processing modules of the system;
• a fourth processing module comprising phoneme-to-viseme mapping means for processing said phoneme stream so as to produce a morph target stream representing a series of morph targets, where a morph target comprises a predefined shape of at least a portion of the face of the character, wherein each said morph target in the series comprises a viseme, where a viseme comprises a predefined shape of at least a mouth portion of the face containing a mouth of the character in a predetermined mouth shape matching a said phoneme;
• a fifth processing module comprising morph target insertion means for modifying said morph target stream, using mark up instructions contained in said mark up instruction stream, so as to produce a modified morph target stream representing a modified series of morph targets comprising at least one further morph target inserted therein;
• and
• a sixth processing module comprising rendering means for processing said modified morph target stream and said mark up instruction stream, so as to generate an animated image of at least the character's head, said animated image comprising a sequence of image frames including image frames showing the character's face in said predefined shapes corresponding to the morph targets in said modified morph target stream;
• and for displaying, on a display means accessible to the system in use thereof, said animated image synchronously with generating the audio signal from said audio signal stream so that the character appears to speak, the movement of the mouth portion of the face matching the phonemes in the audio signal.

A computer program product comprising:

• a computer usable medium having computer readable code means embodied in said medium for generating an animated image of at least the head of a character which is speaking, the character's face having visible articulation matching words being spoken, said computer readable code means comprising:
• program code for processing a first stream of marked up text comprising text to be spoken by an animated character and a plurality of mark up instructions representing behavioural features to be implemented in the animated character, so as to replace recognisable ones of said mark up instructions with predefined modified mark up instructions, so as to convert said first stream of marked up text into a second, modified, stream of marked up text;
• program code for processing said second stream of marked up text using randomising means so as to insert additional randomly selected mark up instructions into said second stream of marked up text, said randomly selected mark up instructions representing random behavioural features to be implemented in the animated character;
• program code for processing said second stream of marked up text so as to produce:
• an audio signal stream for generating an audio signal representing said text being spoken;
• a phoneme stream comprising a sequence of phonemes corresponding to successive portions of the audio signal;
• and a mark up instruction stream comprising mark up instructions for use in other processing phases of the method;
• program code for processing said phoneme stream using phoneme-to-viseme mapping means so as to produce a morph target stream representing a series of morph targets, where a morph target comprises a predefined shape of at least a portion of the face of the character, wherein each said morph target in the series comprises a viseme, where a viseme comprises a predefined shape of at least a mouth portion of the face containing a mouth of the character in a predetermined mouth shape matching a said phoneme;
• program code for modifying said morph target stream using mark-up instructions contained in said mark-up instruction stream so as to produce a modified morph target stream representing a modified series of morph targets comprising said series of visemes having at least one further morph target inserted therein;
• and
• program code for:
• processing said modified morph target stream, using mark-up instructions contained in said mark-up instruction stream, so as to generate an animated image of at least the head of the character, said animated image comprising a sequence of image frames including image frames showing the character's face in said predefined shapes corresponding to the morph targets in the modified morph target stream;
• and displaying said animated image on a display means synchronously with generating said audio signal from said audio signal stream so that the character appears to speak, the movement of the mouth portion of the face matching the phonemes in the audio signal.

A method of generating an animated image of at least a head of a character which is speaking, the character's face having visible articulation match words being spoken, the method comprising:

• a) receiving an output stream of marked up text;
• b) randomly inserting mark-up instructions therein so as to produce:
• an audio signal stream for use in generating an audio signal representing said text being spoken;
• a phoneme signal stream representing a sequence of phonemes corresponding to successive portions of the audio signal;
• and a mark up instruction stream comprising mark up instructions for use in other processing phases of the method;
• c) processing said phoneme stream using phoneme-to-viseme mapping means so as to produce a morph target stream representing a series of morph targets, where a morph target comprises a predefined shape of the head of the character, wherein each said morph target in the series comprises a viseme, where a viseme comprises a predefined shape of the face containing a mouth of the character in a predetermined mouth shape matching a said phoneme;
• d) modifying said morph target stream using mark-up instructions contained in said mark-up instruction stream so as to produce a modified morph target stream representing a modified series of morph targets comprising said series of visemes having at least one further morph target inserted therein;
• e) inserting additional randomly selected morph targets into said modified series of morph targets, said additional morph targets providing random movements/expressions in the animated face;
• f) processing said modified morph target stream, and said mark-up instruction stream, so as to generate an animated image of at least the character's head, said animated image comprising a sequence of image frames including image frames showing the character's face in said predefined shapes corresponding to the morph targets in said modified morph target stream;
• and
• g) displaying said animated image on a display means synchronously with generating said audio signal from said audio signal stream so that the animated character appears to speak, the movement of the mouth portion of the character's face matching the phonemes in the audio signal.