# US 2013/0155063 A1 · Face Feature Vector Construction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0155063 A1
- Document ID: US20130155063A1
- Status: Published application
- Kind: A1
- Application number: 13/332,084
- Filed: 2011-12-20
- Published: 2013-06-20
- Earliest claimed: 2011-12-20
- Assignees: Apple Inc.
- Inventors: Jan Erik Solem; Michael Rousson
- Web page: https://patentgrep.com/patent/US20130155063A1

## CPC

- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06V 10/7557: {based on appearance, e.g. active appearance models [AAM]}
- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, methods, and computer readable media for determining and applying face recognition parameter sets are described. In general, techniques are disclosed for identifying and constructing a unique combination of facial recognition discriminators into a “face feature vector” that has been found to be more robust (e.g., stable to image noise, a person's pose, and scene illumination) and accurate (e.g., provide high recognition rates) than prior art techniques. More particularly, a face feature vector may be generated by the combination of shape descriptors (e.g., as generated by two-dimensional and three-dimensional shape models) and texture descriptors (e.g., as generated by global and local texture models).

## Claims

### Claim 1

A non-transitory computer readable medium, comprising computer code to:
obtain landmark detection information for a first face in a first image;
generate a first shape model feature vector based, at least in part, on the landmark detection information;
generate a second shape model feature vector based, at least in part, on the landmark detection information;
generate a first texture model feature vector based, at least in part, on the landmark detection information;
generate a second texture model feature vector based, at least in part, on the landmark detection information;
combine the first shape model feature vector, the second shape model feature vector, the first texture model feature vector and the second texture model feature vector to form a first face feature vector;
store the first face feature vector in a storage device;
retrieve the first face feature vector from the storage device;
retrieve a second face feature vector form the storage device, wherein the second face feature vector corresponds to a known person;
compare the first face feature vector and the second face feature vector to generate a similarity value;
determine the first face corresponds to the known person if the similarity value is greater than a specified amount; and
determine the first face does not correspond to the known person if the similarity value is not greater than the specified amount.

### Claim 2 (depends on 1)

The non-transitory computer readable medium of claim 1, wherein the computer code to generate a first shape model feature vector comprises computer code to apply the landmark detection information to a two-dimensional model of the first face.

### Claim 3 (depends on 2)

The non-transitory computer readable medium of claim 2, wherein the computer code to generate a second shape model feature vector comprises computer code to apply the landmark detection information to a three-dimensional model of the first face.

### Claim 4 (depends on 1)

The non-transitory computer readable medium of claim 1, further comprising computer code to generate normalized landmark detection information based, at least in part, on the landmark detection information.

### Claim 5 (depends on 4)

The non-transitory computer readable medium of claim 4, wherein the computer code to generate a first texture model feature vector comprises computer code to generate a first texture model feature vector based, at least in part, on the normalized landmark detection information.

### Claim 6 (depends on 5)

The non-transitory computer readable medium of claim 5, wherein the computer code to generate a first texture model feature vector comprises computer code to:
identify a plurality of regions in the normalized landmark detection information, wherein the plurality of regions include less than all of the normalized landmark detection information; and
generate a first texture model feature vector based on the plurality of regions.

### Claim 7 (depends on 5)

The non-transitory computer readable medium of claim 5, wherein the computer code to generate a second texture model feature vector comprises computer code to:
generate warped landmark detection information based, at least in part, on the normalized landmark detection information and a specified morphing operation; and
generate a second texture model feature vector based, at least in part, on the warped landmark detection information.

### Claim 8 (depends on 5)

The non-transitory computer readable medium of claim 5, wherein the computer code generate a second texture model feature vector, further comprises computer code to reduce the dimensionality of the second texture model feature vector before performing the computer code to combine the first texture model feature vector and the second texture model feature vector to form a first face feature vector.

### Claim 9 (depends on 1)

The non-transitory computer readable medium of claim 1, wherein the computer code to obtain landmark detection information comprises computer code to obtain landmark detection information for the first face in the first image from a face detection operation.

### Claim 10 (depends on 1)

The non-transitory computer readable medium of claim 1, wherein the computer code to combine the first shape model feature vector, the second shape model feature vector, the first texture model feature vector and the second texture model feature vector comprises computer code to concatenate the first shape model feature vector, the second shape model feature vector, the first texture model feature vector and the second texture model feature vector.

### Claim 11 (depends on 1)

The non-transitory computer readable medium of claim 1, wherein the computer code to store the first face feature vector in a storage device comprises computer code to:
incorporate the first face feature vector into metadata of the first image; and
store the first image with the first face feature vector in a storage device.

### Claim 12

(canceled)

### Claim 13 (depends on 1)

An electronic device, comprising:
storage device having stored therein a plurality of images;
memory communicatively coupled to the storage device and having stored therein computer code in accordance with claim 1; and
a programmable control unit communicatively coupled to the storage device and the memory and configured to retrieve and execute the program code stored in the memory.

### Claim 14

A non-transitory computer readable medium, comprising computer code to:
obtain a landmark image for a first face in a first image, the landmark image identifying a plurality of aspects of the first face;
generate a normalized landmark image based, at least in part, on the landmark image;
generate a warped landmark image based, at least in part, on the normalized landmark image;
generate a first shape model feature vector based, at least in part, on the landmark image;
generate a second shape model feature vector based, at least in part, on the landmark image;
generate a first texture model feature vector based, at least in part, on the normalized landmark image;
generate a second texture model feature vector based, at least in part, on the warped landmark image;
combine the first shape model feature vector, the second shape model feature vector, the first texture model feature vector and the second texture model feature vector to form a first face feature vector;
store the first face feature vector in a storage device;
retrieve the first face feature vector from the storage device;
retrieve a second face feature vector form the storage device, wherein the second face feature vector corresponds to a known person;
compare the first face feature vector and the second face feature vector to generate a similarity value;
determine the first face corresponds to the known person if the similarity value is greater than a specified amount; and
determine the first face does not correspond to the known person if the similarity value is not greater than the specified amount.

### Claim 15 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the computer code to generate a first shape model feature vector comprises computer code to apply the landmark detection information to a two-dimensional model of the first face.

### Claim 16 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the computer code to generate a second shape model feature vector comprises computer code to apply the landmark detection information to a three-dimensional model of the first face.

### Claim 17 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the computer code to generate a first texture model feature vector comprises computer code to use a gradient vector operation on a plurality of regions in the normalized landmark image, wherein the plurality of regions comprise less than all of the normalized landmark image.

### Claim 18 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the computer code generate a second texture model feature vector, further comprises computer code to:
generate first and second descriptors based, at least in part, on the normalized landmark image;
generate a third descriptor based, at least in part, on the warped landmark image; and
combine the first, second and third descriptors to form the second texture model feature vector.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the computer code to combine the first, second and third descriptors further comprises computer code to reduce the dimensionality of the combined first, second and third descriptors.

### Claim 20 (depends on 14)

AR computer system, comprising:
a storage device having stored thereon a plurality of images;
a memory operatively coupled to the storage device and having stored therein computer code in accordance with claim 14; and
a programmable control device communicatively coupled to the storage device and the memory and configured to execute the computer code stored in the memory.
