# US 7,609,859 B2 · Method and system for generating bi-linear models for faces

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 7,609,859 B2
- Document ID: US7609859B2
- Status: Granted
- Kind: B2
- Application number: 11/152,528
- Filed: 2005-06-14
- Published: 2006-12-14
- Granted: 2009-10-27
- Earliest claimed: 2005-06-14
- Assignees: Mitsubishi Electric Research Laboratories, Inc.
- Inventors: Jinho Lee; Baback Moghaddam; Hanspeter Pfister; Raghu Machiraju
- Web page: https://patentgrep.com/patent/US7609859B2

## CPC

- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method generates a three-dimensional, bi-linear, illumination model for arbitrary faces. A large number of images are acquired of many different faces. For each face, multiple images are acquired with varying poses and varying illumination. A three-mode singular value decomposition is applied to the images to determine parameters of the model. The model can be fit to a probe image of an unknown face. Then, the model can be compared with models of a gallery of images of unknown faces to recognize the face in the probe image.

## Claims

### Claim 1

A method for generating a bi-linear model of faces, comprising a processor for performing steps of the method, comprising the steps of:
acquiring, for each of a plurality of faces, a plurality of 2D images, the plurality of images acquired with a varying poses of the face and varying illuminations on the face;
acquiring, for each of the plurality of faces, a 3D shape model;
obtaining, for each of the plurality of faces, point-to-point correspondences using the 2D images and the 3D shape model;
determining, for each of the plurality of faces, a diffuse texture from the 2D images;
aligning the point-to-point correspondences and the diffuse textures into vectors;
applying a singular value decomposition to the vectors to determine a joint variation of both shape and illumination;
determining shape and illumination basis vectors from the shape and illumination; and
reconstructing corresponding shape and illumination bases from a linear combination of the shape and illumination vectors, and a shape parameter to generate a bi-linear face model.

### Claim 2 (depends on 1)

The method of claim 1, in which the 2D images are high-dynamic range images.

### Claim 3 (depends on 1)

The method of claim 1, in which the 3D shape model is in a form of a polygon mesh.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
converting the polygon mesh to an unstructured list of point samples.

### Claim 5 (depends on 1)

The method of claim 1, in which the singular value decomposition is a three-mode, higher order decomposition.

### Claim 6 (depends on 1)

The method of claim 1 in which the shape and illumination vectors are three-mode tensors.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
fitting the 3D shape model to the 2D images using a linear subspace.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
performing face recognition with the bi-linear face model.