# US 8,787,629 B2 · Image processing based on line-of-sight of a person

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,787,629 B2
- Document ID: US8787629B2
- Status: Granted
- Kind: B2
- Application number: 13/446,898
- Filed: 2012-04-13
- Published: 2012-10-25
- Granted: 2014-07-22
- Earliest claimed: 2011-04-21
- Assignees: Canon Kabushiki Kaisha
- Applicants: Yuji Kaneda; Kotaro Yano
- Inventors: Yuji Kaneda; Kotaro Yano
- Web page: https://patentgrep.com/patent/US8787629B2

## CPC

- G06V 40/19: Sensors therefor
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An information processing apparatus includes a face detection unit configured to detect a face of a person from an image; a storage unit configured to store dictionary data which holds information relating to faces and line-of-sight correction data corresponding to respective persons; an authentication unit configured to authenticate a person corresponding to the face detected by the face detection unit, using the information relating to faces in the dictionary data; a line-of-sight detection unit configured to detect information relating to a line of sight from the face detected by the face detection unit; and a line-of-sight correction unit configured to correct the information relating to a line of sight detected by the line-of-sight detection unit, using the line-of-sight correction data in the dictionary data corresponding to the person authenticated by the authentication unit.

## Claims

### Claim 1

An image capturing apparatus comprising:
a face detection unit configured to detect a face of a person from an image;
a storage unit configured to store dictionary data which holds information relating to faces and line-of-sight correction data corresponding to respective persons;
an authentication unit configured to authenticate a person corresponding to the face detected by the face detection unit, using the information relating to faces in the dictionary data;
a line-of-sight detection unit configured to detect information relating to a line of sight from the face detected by the face detection unit;
a line-of-sight correction unit configured to correct the information relating to a line of sight detected by the line-of-sight detection unit, using the line-of-sight correction data in the dictionary data corresponding to the person authenticated by the authentication unit;
a determination unit configured to determine whether a line of sight of a person serving as an object whose image is to be captured faces toward the image capturing apparatus, based on the information relating to a line of sight that is output from the line-of-sight correction unit; and
an image capturing control unit configured control an execution of image capturing based on a result of the determination with the determination unit.

### Claim 2 (depends on 1)

The image capturing apparatus according to claim 1, further comprising an organ detection unit configured to detect a facial organ from the face detected by the face detection unit; wherein the line-of-sight detection unit detects the information relating to a line of sight using positional information of the facial organ detected by the organ detection unit and averaged positional information of the facial organ.

### Claim 3 (depends on 1)

The image capturing apparatus according to claim 1, wherein the line-of-sight detection unit detects, as the information relating to a line of sight, a direction of the line of sight or a likelihood that a direction of the line of sight matches a predetermined direction.

### Claim 4 (depends on 1)

The image capturing apparatus according to claim 1, further comprising a registration unit configured to generate line-of-sight correction data based on the information relating to a line of sight detected by the line-of-sight detection unit, and register or update the generated line-of-sight correction data in said dictionary data.

### Claim 5 (depends on 4)

The image capturing apparatus according to claim 4, wherein the dictionary data holds weighted average values of line-of-sight correction data corresponding to each of the persons; and the registration unit registers or updates, in the dictionary data, the information relating to a line of sight detected by the line-of-sight detection unit and weighted average values of the line-of-sight correction data in the dictionary data.

### Claim 6 (depends on 4)

The image capturing apparatus according to claim 4, wherein the registration unit carries out the registration or update in the dictionary data if at least one of a state of the face detected by the face detection unit and the information relating to the line of sight detected by the line-of-sight detection unit satisfies a predetermined condition.

### Claim 7 (depends on 6)

The image capturing apparatus according to claim 6, wherein the predetermined condition concerning a state of the face relates to at least one of an orientation of the face, a size of the face and an expression of the face detected by the face detection unit.

### Claim 8

A processing method of an image capturing apparatus, the processing method comprising:
detecting a face of a person from an image;
authenticating a person corresponding to the detected face, using information relating to the face in dictionary data which holds information relating to faces and line-of-sight correction data corresponding to respective persons;
detecting information relating to a line of sight from the detected face;
correcting the information relating to the detected line of sight, using the line-of-sight correction data in the dictionary data corresponding to the authenticated person;
determining whether a line of sight of a person serving as an object whose image is to be captured faces toward the image capturing apparatus, based on the information relating to a line of sight that is corrected; and
controlling an execution of image capturing based on a result of the determination.

### Claim 9

A non-transitory computer-readable storage medium storing a computer program that causes a computer to function as:
a face detection unit configured to detect a face of a person from an image;
a storage unit configured store dictionary data which holds information relating to faces and line-of-sight correction data corresponding to respective persons;
an authentication unit configured to authenticate a person corresponding to the face detected by the face detection unit, using the information relating to faces in the dictionary data;
a line-of-sight detection unit configured to detect information relating to a line of sight from the face detected by the face detection unit;
a line-of-sight correction unit configured to correct the information relating to a line of sight detected by the line-of-sight detection unit, using the line-of-sight correction data in the dictionary data corresponding to the person authenticated by the authentication unit;
a determination unit configured to determine whether a line of sight of a person serving as an object whose image is to be captured faces toward an image capturing apparatus, based on the information relating to a line of sight that is output from the line-of-sight correction unit; and
an image capturing control unit configured control an execution of image capturing based on a result of the determination with the determination unit.