# US 2010/0141787 A1 · Face recognition using face tracker classifier data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0141787 A1
- Document ID: US20100141787A1
- Status: Published application
- Kind: A1
- Application number: 12/631,733
- Filed: 2009-12-04
- Published: 2010-06-10
- Earliest claimed: 2008-12-05
- Assignees: FotoNation Ireland Limited
- Inventors: Petronel Bigioi; Peter Corcoran
- Web page: https://patentgrep.com/patent/US20100141787A1

## CPC

- G06V 40/172: {Classification, e.g. identification}
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of determining face recognition profiles for a group persons includes determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of the group, and recording probability scores which are analyzed for each classifier, including determining a mean and variance for each classifier for the first person. The process is repeated for one or more other persons of the group. A sub-set of classifiers is determined which best differentiates between the first person and the one or more other persons. The sub-set of classifiers is stored in association with the first person as a recognition profile.

## Claims

### Claim 1

A method of in-camera face recognition training of a specific face within digital images acquired with a portable camera-enabled device, comprising:
using a lens, image sensor and processor of a portable camera-enabled device to acquire digital images;
generating in the device, capturing or otherwise obtaining in the device multiple different images that include a face of a specific person;
identifying groups of pixels that correspond to the face within the multiple different images;
selecting sets of classifiers as matching the faces identified in the multiple different images;
statistically analyzing the sets of classifiers to generate a reference classifier profile of the face associated with the specific person;
repeating the generating, identifying, selecting and statistically analyzing one or more times to generate one or more further reference classifier profiles of the face associated with the specific person;
normalizing the reference classifier profiles to determine normalized face classifiers of an average face associated with the specific person;
generating a face recognition profile for the specific person based on the normalized face classifiers of the average face of the specific person; and
programming the processor such that when a probability exceeds a predetermined threshold that an identified face corresponds to the specific person associated with the face recognition profile, then the processor initiates a workflow, image processing or other pre- or post-image acquisition action on the portable camera-enabled device based on the recognition of the face as belonging to the specific person.

### Claim 2 (depends on 1)

The method of claim 1, wherein the different images comprise two or more different poses of the face.

### Claim 3 (depends on 2)

The method of claim 2, wherein the different images comprises two or more different lighting conditions of the face.

### Claim 4 (depends on 1)

The method of claim 1, wherein the different images comprises two or more different lighting conditions of the face.

### Claim 5 (depends on 1)

The method of claim 1, wherein the statistically analyzing comprises determining mean and variance values for the sets of classifiers.

### Claim 6 (depends on 1)

The method of claim 1, further comprising tracking said face within the plurality of the relatively low resolution images.

### Claim 7 (depends on 6)

The method of claim 6, wherein the selecting comprises aligning face region subwindows and determining probabilities for classifiers based upon which the sets of classifiers are selected.

### Claim 8 (depends on 6)

The method of claim 6, wherein the tracking is performed in parallel with determining whether the identified face corresponds to the specific person.

### Claim 9 (depends on 1)

The method of claim 1, further comprising determining probabilities for classifiers based upon which the sets of classifiers are selected.

### Claim 10

A method of determining face recognition profiles for each of a group of at least two persons comprising analyzing a set of images where more than one face image of each person is available, comprising:
(a) determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of a group of persons;
(b) recording a probability score for each classifier stage of the multi-classifier face detector, along with an overall probability score, including associating the probability scores with the first person;
(c) repeating (a) and (b) for one or more further images each containing a face region that has above the threshold probability of corresponding to the first person;
(d) analyzing the probability scores for each classifier and the overall probability score, including determining a mean and variance for each classifier for the first person;
(e) repeating steps (a) through (d) for one or more further persons of the group of persons;
(f) determining a sub-set of classifiers which best differentiates between the first person and the one or more further persons of the group; and
(g) storing the sub-set of classifiers including associating said sub-set of classifiers with the first person as a recognition profile.

### Claim 11 (depends on 10)

The method of claim 10, further comprising deriving the one or more further images each containing a face region of the first person from a set of preview stream images acquired within a same digital imaging device as the digital image.

### Claim 12 (depends on 10)

The method of claim 10, further comprising storing the mean and variance for each classifier and the overall probability score, including associating these with the first person as a base recognition profile.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
determining and storing base recognition profile data for the one or more further persons of the group of persons;
determining a new sub-set of classifiers which best differentiates the first person from the one or more further persons, and
storing the new sub-set of classifiers as a new recognition profile for the first person, including associating the new sub-set of classifiers with the first person.

### Claim 14 (depends on 13)

The method of claim 13, further comprising deriving multiple face images of the first person from a set of preview stream images, and determining the base recognition profile on these multiple face images from the set of preview stream images.

### Claim 15 (depends on 14)

The method of claim 14, further comprising storing the base recognition profile of the first person and the base recognition profile data of the one or more further persons of the group of persons within a same digital imaging device used to capture the digital image and set of preview stream images.

### Claim 16

A portable camera-enabled device capable of in-camera face recognition and face recognition training of a specific face within acquired digital images, comprising:
a lens and an image sensor to acquire digital images;
a processor programmed by processor-readable code embedded within one or more digital storage media, wherein the processor-readable code comprises:
a face detector component to program the processor to identify a groups of pixels that correspond to a face within the digital images;
a face recognition training component to program the processor to generate a face recognition profile for a specific person determined by programming the processor to perform the following:
selecting sets of classifiers as matching the faces identified in the multiple different images;
statistically analyzing the sets of classifiers to generate a reference classifier profile of the face associated with the specific person;
repeating the generating, identifying, selecting and statistically analyzing one or more times to generate one or more further reference classifier profiles of the face associated with the specific person;
normalizing the reference classifier profiles to determine normalized face classifiers of an average face associated with the specific person;
a face recognition component to determine when a probability exceeds a predetermined threshold that an identified face corresponds to the specific person associated with the face recognition profile, and to initiate a workflow, image processing or other pre- or post-image acquisition action on the portable camera-enabled device when the face belonging to the specific person is thereby recognized.

### Claim 17 (depends on 16)

The device of claim 16, wherein the multiple different images comprise two or more different poses of the face.

### Claim 18 (depends on 17)

The device of claim 17, wherein the multiple different images comprises two or more different lighting conditions of the face.

### Claim 19 (depends on 16)

The device of claim 16, wherein the multiple different images comprises two or more different lighting conditions of the face.

### Claim 20 (depends on 16)

The device of claim 16, wherein the multiple statistically analyzing comprises determining mean and variance values for the sets of classifiers.

### Claim 21 (depends on 16)

The device of claim 16, wherein the processor-readable code further comprises a face tracker component to program the processor to track said face within a plurality of images.

### Claim 22 (depends on 21)

The device of claim 21, wherein the selecting comprises aligning face region subwindows and determining probabilities for classifiers based upon which the sets of classifiers are selected.

### Claim 23 (depends on 21)

The device of claim 21, wherein the face tracking and recognition components are configured to operate simultaneously.

### Claim 24 (depends on 16)

The device of claim 16, wherein the processor-readable code further for programming the processor to determine probabilities for classifiers based upon which the sets of classifiers are selected.

### Claim 25

A portable camera-enabled device capable of in-camera determination of face recognition profiles for each of a group of at least two persons when more than one face image of each person is available, comprising:
a lens;
an image sensor;
a processor; and
a processor readable medium having code embedded therein for programming the processor to perform a face recognition method that comprises:
(a) determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of a group of persons;
(b) recording a probability score for each classifier stage of the multi-classifier face detector, along with an overall probability score, including associating the probability scores with the first person;
(c) repeating (a) and (b) for one or more further images each containing a face region that has above the threshold probability of corresponding to the first person;
(d) analyzing the probability scores for each classifier and the overall probability score, including determining a mean and variance for each classifier for the first person;
(e) repeating steps (a) through (d) for one or more further persons of the group of persons;
(f) determining a sub-set of classifiers which best differentiates between the first person and the one or more further persons of the group; and
(g) storing the sub-set of classifiers including associating said sub-set of classifiers with the first person as a recognition profile.

### Claim 26 (depends on 25)

The device of claim 25, wherein the method further comprises deriving the one or more further images each containing a face region of the first person from a set of preview stream images acquired within a same digital imaging device as the digital image.

### Claim 27 (depends on 25)

The device of claim 25, wherein the method further comprises storing the mean and variance for each classifier and the overall probability score, including associating these with the first person as a base recognition profile.

### Claim 28 (depends on 27)

The device of claim 27, wherein the method further comprises:
determining and storing base recognition profile data for the one or more further persons of the group of persons;
determining a new sub-set of classifiers which best differentiates the first person from the one or more further persons, and
storing the new sub-set of classifiers as a new recognition profile for the first person, including associating the new sub-set of classifiers with the first person.

### Claim 29 (depends on 28)

The device of claim 28, wherein the method further comprises deriving multiple face images of the first person from a set of preview stream images, and determining the base recognition profile on these multiple face images from the set of preview stream images.

### Claim 30 (depends on 29)

The device of claim 29, wherein the method further comprises storing the base recognition profile of the first person and the base recognition profile data of the one or more further persons of the group of persons within a same digital imaging device used to capture the digital image and set of preview stream images.

### Claim 31

One or more processor-readable storage media having code embedded therein for programming a processor to perform a method of in-camera face recognition training of a specific face within digital images acquired with a portable camera-enabled device, wherein the method comprises:
using a processor of a portable camera-enabled device;
generating in the device, capturing or otherwise obtaining in the device multiple different images that include a face of a specific person;
identifying groups of pixels that correspond to the face within the multiple different images;
selecting sets of classifiers as matching the faces identified in the multiple different images;
statistically analyzing the sets of classifiers to generate a reference classifier profile of the face associated with the specific person;
repeating the generating, identifying, selecting and statistically analyzing one or more times to generate one or more further reference classifier profiles of the face associated with the specific person;
normalizing the reference classifier profiles to determine normalized face classifiers of an average face associated with the specific person;
generating a face recognition profile for the specific person based on the normalized face classifiers of the average face of the specific person; and
programming the processor such that when a probability exceeds a predetermined threshold that an identified face corresponds to the specific person associated with the face recognition profile, then the processor initiates a workflow, image processing or other pre- or post-image acquisition action on the portable camera-enabled device based on the recognition of the face as belonging to the specific person.

### Claim 32 (depends on 31)

The one or more processor-readable storage media of claim 31, wherein the different images comprise two or more different poses of the face.

### Claim 33 (depends on 32)

The one or more processor-readable storage media of claim 32, wherein the different images comprises two or more different lighting conditions of the face.

### Claim 34 (depends on 31)

The one or more processor-readable storage media of claim 31, wherein the different images comprises two or more different lighting conditions of the face.

### Claim 35 (depends on 31)

The one or more processor-readable storage media of claim 31, wherein the statistically analyzing comprises determining mean and variance values for the sets of classifiers.

### Claim 36 (depends on 31)

The one or more processor-readable storage media of claim 31, wherein the method further comprises tracking said face within the plurality of the relatively low resolution images.

### Claim 37 (depends on 36)

The one or more processor-readable storage media of claim 36, wherein the selecting comprises aligning face region subwindows and determining probabilities for classifiers based upon which the sets of classifiers are selected.

### Claim 38 (depends on 36)

The one or more processor-readable storage media of claim 36, wherein the tracking is performed in parallel with determining whether the identified face corresponds to the specific person.

### Claim 39 (depends on 31)

The one or more processor-readable storage media of claim 31, wherein the method further comprises determining probabilities for classifiers based upon which the sets of classifiers are selected.

### Claim 40

One or more processor-readable storage media having code embedded therein for programming a processor to perform a method of in-camera determination of face recognition profiles for each of a group of at least two persons when more than one face image of each person is available, wherein the method comprises:
(a) determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of a group of persons;
(b) recording a probability score for each classifier stage of the multi-classifier face detector, along with an overall probability score, including associating the probability scores with the first person;
(c) repeating (a) and (b) for one or more further images each containing a face region that has above the threshold probability of corresponding to the first person;
(d) analyzing the probability scores for each classifier and the overall probability score, including determining a mean and variance for each classifier for the first person;
(e) repeating steps (a) through (d) for one or more further persons of the group of persons;
(f) determining a sub-set of classifiers which best differentiates between the first person and the one or more further persons of the group; and
(g) storing the sub-set of classifiers including associating said sub-set of classifiers with the first person as a recognition profile.

### Claim 41 (depends on 40)

The one or more processor-readable storage media of claim 40, wherein the method further comprises deriving the one or more further images each containing a face region of the first person from a set of preview stream images acquired within a same digital imaging device as the digital image.

### Claim 42 (depends on 40)

The one or more processor-readable storage media of claim 40, wherein the method further comprises storing the mean and variance for each classifier and the overall probability score, including associating these with the first person as a base recognition profile.

### Claim 43 (depends on 42)

The one or more processor-readable storage media of claim 42, wherein the method further comprises:
determining and storing base recognition profile data for the one or more further persons of the group of persons;
determining a new sub-set of classifiers which best differentiates the first person from the one or more further persons, and
storing the new sub-set of classifiers as a new recognition profile for the first person, including associating the new sub-set of classifiers with the first person.

### Claim 44 (depends on 43)

The one or more processor-readable storage media of claim 43, wherein the method further comprises deriving multiple face images of the first person from a set of preview stream images, and determining the base recognition profile on these multiple face images from the set of preview stream images.

### Claim 45 (depends on 44)

The one or more processor-readable storage media of claim 44, wherein the method further comprises storing the base recognition profile of the first person and the base recognition profile data of the one or more further persons of the group of persons within a same digital imaging device used to capture the digital image and set of preview stream images.
