# US 11,443,533 B2 · Information processing apparatus and computer readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,533 B2 - Document ID: US11443533B2 - Status: Granted - Kind: B2 - Application number: 16/724,362 - Filed: 2019-12-22 - Published: 2020-07-02 - Granted: 2022-09-13 - Earliest claimed: 2018-12-28 - Assignees: Honda Motor Co., Ltd. - Applicants: Honda Motor Co., Ltd. - Inventors: Yoshikazu Matsuo; Hisumi Esaki; Yusuke Oi - Web page: https://patentgrep.com/patent/US11443533B2 ## CPC - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 40/174: {Facial expression recognition} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 17/00: Speaker identification or verification techniques - G10L 25/63: for estimating an emotional state - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract An information processing apparatus, comprising a determination unit configured to determine whether a vehicle is located inside a lower accuracy area, in which an estimation accuracy of the emotion estimation process lowers, wherein the emotion estimation process is a process for estimating an emotion of an occupant based on an image of the occupant of the vehicle captured by an image-capturing unit provided in the vehicle, and an emotion estimating unit configured to estimate the emotion of an occupant by performing different emotion estimation processes depending on whether the vehicle is located inside the lower accuracy area or not is provided. ## Claims ### Claim 1 An information processing apparatus comprising: a determination unit configured to determine whether a vehicle is located inside a lower accuracy area, in which an estimation accuracy of an emotion estimation process lowers, wherein the emotion estimation process is a process for estimating an emotion of an occupant of the vehicle based on an image of the occupant captured by an image-capturing unit provided in the vehicle; and an emotion estimating unit configured to estimate the emotion of the occupant by performing different emotion estimation processes depending on whether the vehicle is located inside the lower accuracy area or not, wherein the vehicle is located inside the lower accuracy area under a condition that sunlight or shadows interfere with the image of the occupant. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the emotion estimating unit performs an emotion estimation process using the image of the occupant when the vehicle is not located inside the lower accuracy area, and performs an emotion estimation process using information about the occupant other than the image of the occupant when the vehicle is located inside the lower accuracy area. ### Claim 3 (depends on 2) The information processing apparatus according to claim 2, wherein the emotion estimating unit performs an emotion estimation process without using the image of the occupant when the vehicle is located inside the lower accuracy area. ### Claim 4 (depends on 2) The information processing apparatus according to claim 2, wherein the emotion estimating unit performs an emotion estimation process using a voice of the occupant when the vehicle is located inside the lower accuracy area. ### Claim 5 (depends on 4) The information processing apparatus according to claim 4, comprising a speech inducing process executing unit configured to perform a speech inducing process by inducing the occupant to produce a speech when the vehicle is located inside the lower accuracy area. ### Claim 6 (depends on 2) The information processing apparatus according to claim 2, comprising: a request process executing unit configured to perform a request process to request an input of an emotion of the occupant when the vehicle is located inside the lower accuracy area; and an input emotion acquiring unit configured to acquire the emotion of the occupant input in response to the request process, wherein the emotion estimating unit provides the emotion of the occupant acquired by the input emotion acquiring unit as an estimation result of the emotion of the occupant. ### Claim 7 (depends on 2) The information processing apparatus according to claim 2, wherein the emotion estimating unit stores an emotion estimated from the image of the occupant in association with a driving situation of the vehicle when the vehicle is not located inside the lower accuracy area, acquires the driving situation of the vehicle when the vehicle is located inside the lower accuracy area, and provides the emotion stored in association with the acquired driving situation as an estimation result of the emotion of the occupant. ### Claim 8 (depends on 1) The information processing apparatus according to claim 1, wherein the emotion estimating unit estimates an emotion of the occupant to be expressed when the vehicle is located inside the lower accuracy area based on an emotion of the occupant estimated before the vehicle enters the lower accuracy area and an emotion of the occupant estimated after the vehicle leaves the lower accuracy area. ### Claim 9 (depends on 8) The information processing apparatus according to claim 8, comprising: an event determination unit configured to determine whether a preset event occurs while the vehicle drives inside the lower accuracy area, wherein when it is determined that the preset event does not occur while the vehicle drives inside the lower accuracy area, the emotion estimating unit estimates an emotion of the occupant of the vehicle located inside the lower accuracy area based on an emotion of the occupant estimated before the vehicle enters the lower accuracy area and an emotion of the occupant estimated after the vehicle leaves the lower accuracy area. ### Claim 10 (depends on 1) The information processing apparatus according to claim 1, wherein the emotion estimating unit performs an emotion estimation process using the image of the occupant and a voice of the occupant when the vehicle is not located inside the lower accuracy area, and performs an emotion estimation process with decreased weighting when the vehicle is located inside the lower accuracy area, wherein the decreased weighting applies a smaller weight to the image of the occupant than in the emotion estimation process performed when the vehicle is not located inside the lower accuracy area. ### Claim 11 (depends on 10) The information processing apparatus according to claim 10, comprising a speech inducing process executing unit configured to perform a speech inducing process to induce the occupant to produce a speech, when the vehicle is located inside the lower accuracy area. ### Claim 12 (depends on 1) The information processing apparatus according to claim 1, wherein the determination unit further determines an occupant for whom an estimation accuracy of an image-based emotion estimation process lowers and an occupant for whom the estimation accuracy of the image-based emotion estimation process does not lower among a plurality of occupants of the vehicle when the vehicle is located inside the lower accuracy area, and the emotion estimating unit performs an emotion estimation process to estimate the emotion of the occupant for whom the estimation accuracy lowers based on an emotion estimated using the image of the occupant for whom the estimation accuracy does not lower when the vehicle is located inside the lower accuracy area. ### Claim 13 (depends on 12) The information processing apparatus according to claim 12, wherein the emotion estimating unit stores, in association with each other, the emotions of a plurality of occupants of the vehicle estimated using images of the plurality of occupants when the vehicle is not located inside the lower accuracy area, estimates the emotion of the occupant for whom the estimation accuracy does not lower using the image of the occupant when the vehicle is located inside the lower accuracy area, and provides an emotion of the occupant for whom the estimation accuracy lowers which is stored in association with the estimated emotion of the occupant for whom the estimation accuracy does not lower as an estimation result of the emotion of the occupant for whom the estimation accuracy lowers. ### Claim 14 A non-transitory computer readable storage medium having stored thereon a program for causing a computer to serve as: a determination unit configured to determine, whether a vehicle is located inside a lower accuracy area, in which an estimation accuracy of an emotion estimation process lowers, wherein the emotion estimation process is a process for estimating an emotion of an occupant based on an image of the occupant of the vehicle captured by an image-capturing unit provided in the vehicle; and an emotion estimating unit configured to estimate the emotion of the occupant by performing different emotion estimation processes depending on whether the vehicle is located inside the lower accuracy area or not, wherein the vehicle is located inside the lower accuracy area under a condition that sunlight or shadows interfere with the image of the occupant. ### Claim 15 An information processing apparatus comprising: a determination unit configured to determine whether a vehicle is located inside a lower accuracy area, in which an estimation accuracy of an emotion estimation process lowers, wherein the emotion estimation process is a process for estimating an emotion of an occupant of the vehicle based on an image of the occupant captured by an image-capturing unit provided in the vehicle; and an emotion estimating unit configured to estimate the emotion of the occupant by performing different emotion estimation processes depending on whether the vehicle is located inside the lower accuracy area or not, and estimates the emotion of the occupant based on the emotion of the occupant while inside the lower accuracy area and the emotion of the occupant while outside the lower accuracy area.