Computer generated head

A method of adapting a system for rendering a computer generated face to a new expression, the method comprising:

• receiving text data related to speech, which is to be output by movement of the mouth;
• dividing the text data into a sequence of acoustic units including one of at least phonemes, graphemes, and words or parts of words;
• analyzing the text data related to the speech to identify expression-dependent weightings related to a speech expression and a corresponding facial expression, to be input into a statistical model;
• converting the sequence of acoustic units into a sequence of image vectors and a sequence of speech vectors using the statistical model, wherein the model has a plurality of model parameters comprising mathematical means of probability distributions, which relate an acoustic unit in the sequence of acoustic units to an image vector in the sequence of image vectors and to a speech vector in the sequence of speech vectors, the image vector including a plurality of parameters that define the face;
• outputting the sequence of image vectors and the sequence of speech vectors,
• wherein the sequence of image vectors are output as video such that the mouth moves to mime the speech expression associated with the corresponding facial expression, and
• wherein the sequence of speech vectors are output as audio, which is synchronized with lip movement of the mouth and is associated with the speech expression,
• wherein the mathematical means of each probability distribution of the probability distributions for the speech expression and the corresponding facial expression are expressed as a weighted sum of independent mathematical means,
• wherein weightings used in the weighted sum are the identified expression-dependent weightings,
• wherein the independent mathematical means are provided in clusters, and
• wherein here is one expression-dependent weighting per cluster,
• receiving a video file associated with the new expression;
• and
• calculating the identified expression-dependent weightings to weigh parameters of a same type in order to maximize a similarity between the computer generated face and the new expression.

A system for animating a computer generation of a face having a mouth, the system comprising:

• a text input configured to receive data related to speech, which is to be output by movement of the mouth;
• and
• a processor configured to:
• divide the received data into a sequence of acoustic units including one of at least phonemes, graphemes, and words or parts of words;
• analyze the received data related to the speech to identify expression-dependent weightings related to a speech expression and a corresponding facial expression, to be input into a statistical model;
• convert the sequence of acoustic units into a sequence of image vectors and a sequence of speech vectors using the statistical model, wherein the model has a plurality of model parameters comprising mathematical means of probability distributions, which relate an acoustic unit in the sequence of acoustic units to an image vector in the sequence of image vectors and to a speech vector in the sequence of speech vectors, the age vector including a plurality of parameters that define the face;
• and
• output the sequence of image vectors and the sequence of speech vectors,
• wherein the sequence of image vectors are output as video such that the mouth moves to mime the speech expression associated with the corresponding facial expression, and
• wherein the sequence of speech vectors are output as audio, which is synchronized with lip movement of the mouth and is associated with the speech expression,
• wherein the mathematical means of each probability distribution of the probability distributions for the speech expression and the corresponding facial expression are expressed as a weighted sum of independent mathematical means,
• wherein weightings used in the weighted sum are the identified expression-dependent weightings
• wherein the independent mathematical means are provided in clusters, and
• wherein there is one expression-dependent weighting per cluster.

An adaptable system for rendering a computer generated face to a new expression, the system comprising:

• a text input configured to receive data related to speech, which is to be output by movement of the mouth;
• a processor configured to:
• divide the received data into a sequence of acoustic units including one of at least phonemes, graphemes, and words or parts of words;
• analyze the received data related to the speech to identify expression-dependent weightings related to a speech expression and a corresponding facial expression, to be input into a statistical model;
• convert the sequence of acoustic units into a sequence of image vectors and a sequence of speech vectors using the statistical model, wherein the model has a plurality of model parameters comprising mathematical means of probability distributions, which relate an acoustic unit in the sequence of acoustic units to an image vector in the sequence of image vectors and to a speech vector in the sequence of speech vectors, the image vector including a plurality of parameters that define the face;
• output the sequence of image vectors and the sequence of speech vectors,
• wherein the sequence of image vectors are output as video such that the mouth moves to mime the speech expression associated with the corresponding facial expression, and
• wherein the sequence of speech vectors are output as audio, which is synchronized with lip movement of the mouth and is associated with the speech expression,
• wherein the mathematical means of each probability distribution of the probability distributions for the speech expression and the corresponding facial expression are expressed as a weighted sum of independent mathematical means,
• wherein weightings used in the weighted sum are the identified expression-dependent weightings,
• wherein the independent mathematical means are provided in clusters, and
• wherein there is one expression-dependent weighting per cluster;
• receive a video file associated with the new expression;
• and
• calculate the identified expression-dependent weightings to weigh parameters of a same type in order to maximize a similarity between the computer generated face and the new expression.