# US 2005/0089199 A1 · Facial image processing methods and systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2005/0089199 A1
- Document ID: US20050089199A1
- Status: Published application
- Kind: A1
- Application number: 10/980,073
- Filed: 2004-11-02
- Published: 2005-04-28
- Earliest claimed: 2000-08-30
- Assignees: Microsoft Corporation
- Inventors: Stephen Marschner; Brian Guenter; Sashi Raghupathy; Kirk Olynyk; Sing Kang
- Web page: https://patentgrep.com/patent/US20050089199A1

## CPC

- G06T 15/506: {Illumination models}
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V 40/168: {Feature extraction; Face representation}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

In the described embodiment, methods and systems for processing facial image data for use in animation are described. In one embodiment, a system is provided that illuminates a face with illumination that is sufficient to enable the simultaneous capture of both structure data, e.g. a range or depth map, and reflectance properties, e.g. the diffuse reflectance of a subject's face. This captured information can then be used for various facial animation operations, among which are included expression recognition and expression transformation.

## Claims

### Claim 1

A facial image-processing system comprising:
means for illuminating a face with multiple different light sources, wherein at least one of the light sources is polarized;
means for measuring range map data from said illuminating;
means for measuring image data from said illuminating;
means for deriving a 3-dimensional surface from the range map data;
means for computing surface normals to the 3-dimensional surface; and
means for processing the surface normals and the image data to derive an albedo map.

### Claim 2 (depends on 1)

The system of claim 1, wherein at least one of the light sources is infrared.

### Claim 3 (depends on 1)

The system of claim 1, wherein the light sources are at different frequencies.

### Claim 4

A facial image-processing system comprising:
means for illuminating a face with multiple different light sources, wherein all of the light sources are polarized;
means for measuring range map data from said illuminating;
means for measuring image data from said illuminating;
means for deriving a 3-dimensional surface from the range map data;
means for computing surface normals to the 3-dimensional surface; and
means for processing the surface normals and the image data to derive an albedo map.

### Claim 5

A facial image-processing system comprising:
means for illuminating a face with multiple different light sources;
means for measuring range map data from said illuminating;
means for applying a generic face template to the range map data to reject noise that is associated with the range map data;
means for measuring image data from said illuminating;
means for deriving a 3-dimensional surface from the range map data;
means for computing surface normals to the 3-dimensional surface; and
means for processing the surface normals and the image data to derive an albedo map.

### Claim 6

A facial image-processing system comprising:
means for illuminating a face with multiple different light sources;
means for measuring range map data from said illuminating;
means for measuring image data from said illuminating;
means for filtering the range map data;
means for deriving a 3-dimensional surface from the range map data;
means for computing surface normals to the 3-dimensional surface; and
means for processing the surface normals and the image data to derive an albedo map.

### Claim 7

A facial image-processing system comprising:
one or more processors;
one or more computer-readable media;
computer-readable instructions on the one or more computer-readable media which, when executed by the one or more processors, cause the one or more processors to implement a method comprising:
illuminating a face with multiple different light sources, wherein at least one of the light sources is polarized;
measuring range map data from said illuminating;
measuring image data from said illuminating;
deriving a 3-dimensional surface from the range map data;
computing surface normals to the 3-dimensional surface; and
processing the surface normals and the image data to derive an albedo map.

### Claim 8 (depends on 7)

The system of claim 7, wherein at least one of the light sources is infrared.

### Claim 9 (depends on 7)

The system of claim 7, wherein the light sources are at different frequencies.

### Claim 10

A facial image-processing system comprising:
one or more processors;
one or more computer-readable media;
computer-readable instructions on the one or more computer-readable media which, when executed by the one or more processors, cause the one or more processors to implement a method comprising:
illuminating a face with multiple different light sources, wherein all of the light sources are polarized;
measuring range map data from said illuminating;
measuring image data from said illuminating;
deriving a 3-dimensional surface from the range map data;
computing surface normals to the 3-dimensional surface; and
processing the surface normals and the image data to derive an albedo map.

### Claim 11

A facial image-processing system comprising:
one or more processors;
one or more computer-readable media;
computer-readable instructions on the one or more computer-readable media which, when executed by the one or more processors, cause the one or more processors to implement a method comprising:
illuminating a face with multiple different light sources;
measuring range map data from said illuminating;
applying a generic face template to the range map data to reject noise that is associated with the range map data;
measuring image data from said illuminating;
deriving a 3-dimensional surface from the range map data;
computing surface normals to the 3-dimensional surface; and
processing the surface normals and the image data to derive an albedo map.

### Claim 12

A facial image-processing system comprising:
one or more processors;
one or more computer-readable media;
computer-readable instructions on the one or more computer-readable media which, when executed by the one or more processors, cause the one or more processors to implement a method comprising:
illuminating a face with multiple different light sources;
measuring range map data from said illuminating;
measuring image data from said illuminating;
filtering the range map data;
deriving a 3-dimensional surface from the range map data;
computing surface normals to the 3-dimensional surface; and
processing the surface normals and the image data to derive an albedo map.
