US 6,580,811 B2Granted

Wavelet-based facial motion capture for avatar animation

AssigneeEyematic Interfaces, Inc.

InventorsThomas Maurer, Egor Valerievich Elagin, Luciano Pasquale Agostino Nocera, Johannes Bernhard Steffens, Hartmut Neven

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

The present invention is embodied in an apparatus, and related method, for sensing a person's facial movements, features and characteristics and the like to generate and animate an avatar image based on facial sensing. The avatar apparatus uses an image processing technique based on model graphs and bunch graphs that efficiently represent image features as jets. The jets are composed of wavelet transforms processed at node or landmark locations on an image corresponding to readily identifiable features. The nodes are acquired and tracked to animate an avatar image in accordance with the person's facial movements. Also, the facial sensing may use jet similarity to determine the person's facial features and characteristic thus allows tracking of a person's natural characteristics without any unnatural elements that may interfere or inhibit the person's natural characteristics.

Claims12

Dependent claims

A method for generating a head model specific to a person for use in animating a display image having the person's features, the method comprising the steps for:

providing a frontal facial image of the person, the frontal facial image having an array of pixels;
providing a profile facial image of the person, the profile facial image having an array of pixels;
transforming the frontal facial image using wavelet transformations to generate a transformed frontal image having an array of pixels, each pixel of the transformed frontal image being associated with a respective pixel of the frontal facial image and being represented by a frontal transform jet associated with a predetermined number of wavelet component values;
transforming the profile facial image using wavelet transformations to generate a transformed profile image having an array of pixels, each pixel of the transformed profile image being associated with a respective pixel of the profile facial image and being represented by a profile transform jet associated with a predetermined number of wavelet component values;
locating predetermined facial features in the transformed frontal image based on a comparison between the frontal transform jets of the transformed frontal facial image and frontal graph jets at sensing nodes of a frontal graph;
locating predetermined facial features in the transformed profile image based on a comparison between the profile transform jets of the transformed profile facial image and sensing nodes at profile graph jets of a profile graph;
producing a three-dimensional model of the person's head using the frontal facial image and the profile facial image and based on the positions of the predetermined features located in the transformed frontal facial image and the predetermined features located in the transformed profile facial image;
displaying the head model as a display image.

5 more elements

A method for providing a display avatar having a specific person's features, the method comprising the steps for:

providing a frontal facial image of the person, the frontal facial image having an array of pixels;
providing a profile facial image of the person, the profile facial image having an array of pixels;
transforming the frontal facial image using wavelet transformations to generate a transformed frontal image having an array of pixels, each pixel of the transformed frontal image being associated with a respective pixel of the frontal facial image and being represented by a frontal transform jet associated with a predetermined number of wavelet component values;
transforming the profile facial image using wavelet transformations to generate a transformed profile image having an array of pixels, each pixel of the transformed profile image being associated with a respective pixel of the profile facial image and being represented by a profile transform jet associated with a predetermined number of wavelet component values;
locating predetermined facial features in the transformed frontal facial image based on a comparison between the frontal transform jets of the transformed frontal facial image and frontal graph jets at sensing nodes of a frontal graph;
locating predetermined facial features in the transformed profile facial image based on a comparison between the profile transform jets of the transformed profile facial image and sensing nodes at profile graph jets of a profile graph;
producing the display avatar using the frontal facial image and the profile facial image and based on the positions of the predetermined features located in the transformed frontal facial image and the predetermined features located in the transformed profile facial image;
displaying the display avatar.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 09/871,370Filed 31 May 2001 · continuation
Priority: 13 Apr 1998
Published: US 2001/0033675 A125 Oct 2001
Granted: 17 Jun 2003
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

G06T 7/246
Analysis of motion: using feature-based methods, e.g. the tracking of corners or segments
G06T 7/262
Analysis of motion: using transform domain methods, e.g. Fourier domain methods
G06V 10/426
For representing the structure of the pattern or shape of an object therefor: graphical representations
G06V 10/449
By matching or filtering: biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters

Family

Application60/081,615Us provisional application · filed 13 Apr 1998
Application09/188,079Parent of this continuation · filed 6 Nov 1998
US 2001/0033675 A1Pre-grant publication · published 25 Oct 2001
US 6,272,231 B1Related patent
US 6,580,811 B2This patent

From the US filing history. Foreign family members aren’t listed.