# US 12,217,539 B2 · Estimation device, estimation method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,539 B2 - Document ID: US12217539B2 - Status: Granted - Kind: B2 - Application number: 17/680,853 - Filed: 2022-02-25 - Published: 2022-06-09 - Granted: 2025-02-04 - Earliest claimed: 2022-02-25 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Yusuke Morishita - Web page: https://patentgrep.com/patent/US12217539B2 ## CPC - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - A61B 5/163: {by tracking eye movement, gaze, or pupil change} - G06T 7/73: using feature-based methods - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 40/168: {Feature extraction; Face representation} - G06V 40/197: {Matching; Classification} - A61B 5/18: for vehicle drivers {or machine operators} - G06T 2207/30201: Face - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An estimation device according to one aspect of the present disclosure includes: at least one memory storing a set of instructions; and at least one processor configured to execute the set of instructions to: generate a plurality of extraction regions by adding a perturbation to an extraction region of a partial image determined based on positions of feature points extracted from a face image; estimate a plurality of directions of at least one of a face and a line of sight and a reliability of each of the plurality of directions based on a plurality of partial images in the plurality of extraction regions of the face image; and calculate an integrated direction obtained by integrating the plurality of directions based on the estimated reliability. ## Claims ### Claim 1 An estimation device comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to: generate a perturbation extraction region by adding a perturbation to an extraction region in a face image, the perturbation includes a parallel translation of the extraction region, the extraction region being determined based on positions of feature points extracted from the face image, a magnitude of the perturbation being based on an image quality of the face image; estimate, based on a plurality of partial images including a first partial image and a second partial image, a plurality of directions and a reliability of each of the plurality of directions, the plurality of directions being a plurality of face directions or the plurality of directions of lines of sight, the first partial image being extracted from the extraction region in the face image, the second partial image being extracted from the perturbation extraction region in the face image; and calculate an integrated direction obtained by integrating the plurality of directions based on the estimated reliability, wherein the second partial image is generated by adding a second perturbation to an extracted partial image extracted from the perturbation extraction region in the face image, the second perturbation including enlargement of the extracted partial image, the extracted partial image being a third partial image and a fourth partial image. ### Claim 2 (depends on 1) The estimation device according to claim 1, wherein the image quality is based on a magnitude of a contrast of the face image. ### Claim 3 (depends on 1) The estimation device according to claim 1, wherein the at least one processor is further configured to execute the set of instructions to: determine, based on the positions of the feature points, the perturbation added to the extraction region determined based on the positions of the feature points. ### Claim 4 (depends on 1) The estimation device according to claim 1, wherein the at least one processor is further configured to execute the set of instructions to: extract a face region that is a region of the face from the face image; and extract the feature points from the face region. ### Claim 5 (depends on 1) The estimation device according to claim 1, wherein the at least one processor is further configured to execute the set of instructions to: acquire an input image and extracting the face image from the input image; extract the feature points from the face image; and output the integrated direction. ### Claim 6 (depends on 5) The estimation device according to claim 5, wherein the at least one processor is further configured to execute the set of instructions to: estimate the plurality of directions of the face and the plurality of directions of the line of sight; calculate an integrated face direction obtained by integrating the plurality of directions of the face and an integrated line of sight direction obtained by integrating the plurality of directions of the line of sight; superimpose a first mark indicating the integrated face direction and a second mark indicating the integrated line of sight direction on the input image; and output the input image on which the first mark and the second mark are superimposed. ### Claim 7 An estimation method comprising: generating a perturbation extraction region by adding a perturbation to an extraction region in a face image, the perturbation includes a parallel translation of the extraction region, the extraction region being determined based on positions of feature points extracted from the face image, a magnitude of the perturbation being based on an image quality of the face image; estimating, based on a plurality of partial images including a first partial image and a second partial image, a plurality of directions and a reliability of each of the plurality of directions, the plurality of directions being a plurality of face directions or the plurality of directions of lines of sight, the first partial image being extracted from the extraction region in the face image, the second partial image being extracted from the perturbation extraction region in the face image; and calculating an integrated direction obtained by integrating the plurality of directions based on the estimated reliability, wherein the second partial image is generated by adding a second perturbation to an extracted partial image extracted from the perturbation extraction region in the face image, the second perturbation including enlargement of the extracted partial image. ### Claim 8 (depends on 7) The estimation method according to claim 7, wherein the image quality is based on a magnitude of a contrast of the face image. ### Claim 9 (depends on 7) The estimation method according to claim 7, further comprising determining, based on the positions of the feature points, the perturbation added to the extraction region determined based on the positions of the feature points. ### Claim 10 (depends on 7) The estimation method according to claim 7, further comprising: extracting a face region that is a region of the face from the face image; and extracting the feature points from the face region. ### Claim 11 (depends on 7) The estimation method according to claim 7, further comprising: acquiring an input image and extracting the face image from the input image; extracting the feature points from the face image; and outputting the integrated direction. ### Claim 12 (depends on 11) The estimation method according to claim 11, further comprising: estimating the plurality of directions of the face and the plurality of directions of the line of sight; calculating an integrated face direction obtained by integrating the plurality of directions of the face and an integrated line of sight direction obtained by integrating the plurality of directions of the line of sight; superimposing a first mark indicating the integrated face direction and a second mark indicating the integrated line of sight direction on the input image; and outputting the input image on which the first mark and the second mark are superimposed. ### Claim 13 A non-transitory computer readable storage medium storing a program causing a computer to execute processing of: generating a perturbation extraction region by adding a perturbation to an extraction region in a face image, the perturbation includes a parallel translation of the extraction region, the extraction region being determined based on positions of feature points extracted from the face image, a magnitude of the perturbation being based on an image quality of the face image; estimating, based on a plurality of partial images including a first partial image and a second partial image, a plurality of directions and a reliability of each of the plurality of directions, the plurality of directions being a plurality of face directions or the plurality of directions of lines of sight, the first partial image being extracted from the extraction region in the face image, the second partial image being extracted from the perturbation extraction region in the face image; and calculating an integrated direction obtained by integrating the plurality of directions based on the estimated reliability, wherein the second partial image is generated by adding a second perturbation to an extracted partial image extracted from the perturbation extraction region in the face image, the second perturbation including enlargement of the extracted partial image. ### Claim 14 (depends on 13) The storage medium according to claim 13, wherein the image quality is based on a magnitude of a contrast of the face image. ### Claim 15 (depends on 13) The storage medium according to claim 13, wherein the program further causes a computer to execute processing of: determining, based on the positions of the feature points, the perturbation added to the extraction region determined based on the positions of the feature points. ### Claim 16 (depends on 13) The storage medium according to claim 13, wherein the program further causes a computer to execute processing of: extracting a face region that is a region of the face from the face image; and extracting the feature points from the face region. ### Claim 17 (depends on 13) The storage medium according to claim 13, wherein the program further causes a computer to execute processing of: acquiring an input image and extracting the face image from the input image; extracting the feature points from the face image; and outputting the integrated direction. ### Claim 18 (depends on 1) The estimation device according to claim 1, wherein the second partial image is generated by adding a second perturbation to an extracted partial image extracted from the perturbation extraction region in the face image, the second perturbation including a rotation of the extracted partial image.