# US 2022/0036581 A1 · Estimation device, estimation method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036581 A1
- Document ID: US20220036581A1
- Status: Published application
- Kind: A1
- Application number: 17/278,500
- Filed: 2018-09-26
- Published: 2022-02-03
- Earliest claimed: 2018-09-26
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Yusuke MORISHITA
- Web page: https://patentgrep.com/patent/US20220036581A1

## CPC

- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- A61B 5/163: {by tracking eye movement, gaze, or pupil change}
- G06T 7/73: using feature-based methods
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/197: {Matching; Classification}
- A61B 5/18: for vehicle drivers {or machine operators}
- G06T 2207/30201: Face
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An estimation device according to one aspect of the present disclosure includes: at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to: generate a plurality of extraction regions by adding a perturbation to an extraction region of a partial image determined based on positions of feature points extracted from a face image; estimate a plurality of directions of at least one of a face and a line of sight and a reliability of each of the plurality of directions based on a plurality of partial images in the plurality of extraction regions of the face image; and calculate an integrated direction obtained by integrating the plurality of directions based on the estimated reliability.

## Claims

### Claim 1

An estimation device comprising:
at least one memory storing a set of instructions; and
at least one processor configured to execute the set of instructions to:
generate a plurality of extraction regions by adding a perturbation to an extraction region of a partial image determined based on positions of feature points extracted from a face image;
estimate a plurality of directions of at least one of a face and a line of sight and a reliability of each of the plurality of directions based on a plurality of partial images in the plurality of extraction regions of the face image; and
calculate an integrated direction obtained by integrating the plurality of directions based on the estimated reliability.

### Claim 2 (depends on 1)

The estimation device according to claim 1, wherein
the at least one processor is configured to execute the set of instructions to
determine, based on the positions of the feature points, the perturbation added to the extraction region determined based on the positions of the feature points.

### Claim 3 (depends on 1)

The estimation device according to claim 1, wherein
the at least one processor is configured to execute the set of instructions to
extract a face region that is a region of the face from the face image;
extract the feature points from the face region;
estimate a size of the face based on the positions of the extracted feature points, points; and
determine the perturbation based on the estimated size.

### Claim 4 (depends on 1)

The estimation device according to claim 1, wherein
the perturbation is at least one of a change in a size of the extraction region, a change in a position of the extraction region, a change in an angle of the extraction region, and image processing on a partial image extracted from the extraction region.

### Claim 5 (depends on 1)

The estimation device according to claim 1, wherein
the at least one processor is configured to execute the set of instructions to:
acquire an input image and extracting the face image from the input image;
extract the feature points from the face image; and
output the integrated direction.

### Claim 6 (depends on 5)

The estimation device according to claim 5, wherein
the at least one processor is configured to execute the set of instructions to:
estimate a plurality of directions of the face and a plurality of directions of the line of sight;
calculate an integrated face direction obtained by integrating the plurality of directions of the face and an integrated line of sight direction obtained by integrating the plurality of directions of the line of sight;
superimpose a first mark indicating the integrated face direction and a second mark indicating the integrated line of sight direction on the input image; and
output the input image on which the first mark and the second mark are superimposed.

### Claim 7

An estimation method comprising:
generating a plurality of extraction regions by adding a perturbation to an extraction region of a partial image determined based on positions of feature points extracted from a face image;
estimating a plurality of directions of at least one of a face and a line of sight and a reliability of each of the plurality of directions based on a plurality of partial images in the plurality of extraction regions of the face image; and
calculating an integrated direction obtained by integrating the plurality of directions based on the estimated reliability.

### Claim 8 (depends on 7)

The estimation method according to claim 7, further comprising
determining, based on the positions of the feature points, the perturbation added to the extraction region determined based on the positions of the feature points.

### Claim 9 (depends on 7)

The estimation method according to claim 7, further comprising:
extracting a face region that is a region of the face from the face image;
extracting the feature points from the face region;
estimating a size of the face based on the positions of the extracted feature points; and
determining the perturbation based on the estimated size.

### Claim 10 (depends on 7)

The estimation method according to claim 7, wherein
the perturbation is at least one of a change in a size of the extraction region, a change in a position of the extraction region, a change in an angle of the extraction region, and image processing on a partial image extracted from the extraction region.

### Claim 11 (depends on 7)

The estimation method according to claim 7, further comprising:
acquiring an input image and extracting the face image from the input image;
extracting the feature points from the face image; and
outputting the integrated direction.

### Claim 12 (depends on 11)

The estimation method according to claim 11, further comprising:
estimating a plurality of directions of the face and a plurality of directions of the line of sight;
calculating an integrated face direction obtained by integrating the plurality of directions of the face and an integrated line of sight direction obtained by integrating the plurality of directions of the line of sight;
superimposing a first mark indicating the integrated face direction and a second mark indicating the integrated line of sight direction on the input image; and
outputting the input image on which the first mark and the second mark are superimposed.

### Claim 13

A non-transitory computer readable storage medium that stores a program causing a computer to execute:
perturbation processing of generating a plurality of extraction regions by adding a perturbation to an extraction region of a partial image determined based on positions of feature points extracted from a face image;
estimation processing of estimating a plurality of directions of at least one of a face and a line of sight and a reliability of each of the plurality of directions based on a plurality of partial images in the plurality of extraction regions of the face image; and
integration processing of calculating an integrated direction obtained by integrating the plurality of directions based on the estimated reliability.

### Claim 14 (depends on 13)

The storage medium according to claim 13, wherein
the perturbation processing determines, based on the positions of the feature points, the perturbation added to the extraction region determined based on the positions of the feature points.

### Claim 15 (depends on 13)

The storage medium according to claim 13, wherein
the perturbation processing extracts a face region that is a region of the face from the face image, extracts the feature points from the face region, estimates a size of the face based on the positions of the extracted feature points, and determines the perturbation based on the estimated size.

### Claim 16 (depends on 13)

The storage medium according to claim 13, wherein
the perturbation is at least one of a change in a size of the extraction region, a change in a position of the extraction region, a change in an angle of the extraction region, and image processing on a partial image extracted from the extraction region.

### Claim 17 (depends on 13)

The storage medium according to claim 13, the program further causing a computer to execute:
acquisition processing of acquiring an input image and extracting the face image from the input image;
extraction processing of extracting the feature points from the face image; and
output processing of outputting the integrated direction.

### Claim 18 (depends on 17)

The storage medium according to claim 17, wherein
the estimation processing estimates a plurality of directions of the face and a plurality of directions of the line of sight,
the integration processing calculates an integrated face direction obtained by integrating the plurality of directions of the face and an integrated line of sight direction obtained by integrating the plurality of directions of the line of sight, and
the output processing superimposes a first mark indicating the integrated face direction and a second mark indicating the integrated line of sight direction on the input image and outputs the input image on which the first mark and the second mark are superimposed.
