# US 11,077,859 B2 · State determination device, state determination program, and computer-readable non-transitory tangible storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,077,859 B2 - Document ID: US11077859B2 - Status: Granted - Kind: B2 - Application number: 17/026,059 - Filed: 2020-09-18 - Published: 2021-01-07 - Granted: 2021-08-03 - Earliest claimed: 2018-03-23 - Assignees: Denso Corporation - Applicants: Denso Corporation - Inventors: Yukiyasu Yoshimura; Hiroshi Morimoto - Web page: https://patentgrep.com/patent/US11077859B2 ## CPC - B60W 40/08: related to drivers or passengers - B60W 40/09: Driving style or behaviour - G06F 18/254: {of classification results, e.g. of results related to same input data} - G06F 18/256: {of results relating to different input data, e.g. multimodal recognition} - G06V 10/811: {the classifiers operating on different input data, e.g. multi-modal recognition} - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - B60W 2040/0818: {Inactivity or incapacity of driver} - B60W 2040/0872: {Driver physiology} - B60W 2540/221: Physiology, e.g. weight, heartbeat, health or special needs - B60W 2540/223: Posture, e.g. hand, foot, or seat position, turned or inclined - B60W 2540/225: Direction of gaze - G06V 40/16: Human faces, e.g. facial parts, sketches or expressions - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A state determination device to determine that a driver driving a vehicle is in a driving difficult state detects an abnormality in a state of the driver, and detects at least one of an abnormality in a driving operation input by the driver or an abnormality in a traveling state of the vehicle. In response to detecting the abnormality in the state of the driver alone, the state determination device does not determine that the driver is in the driving difficult state but determines that the driver is in the driving difficult state in response to detecting the abnormality in the state of the driver and additionally detecting the at least one of the abnormality in the driving operation input by the driver or the abnormality in the traveling state of the vehicle. ## Claims ### Claim 1 A state determination device to determine that a driver who is driving a vehicle is in a driving difficult state, comprising: a driver abnormality detection unit that detects an abnormality in a state of the driver; an operation abnormality detection unit that detects an abnormality in a driving operation input by the driver; a traveling abnormality detection unit that detects an abnormality in a traveling state of the vehicle; and an integrated determination unit that, in response to only the driver abnormality detection unit detecting the abnormality in the state of the driver alone, does not determine that the driver is in the driving difficult state but determines that the driver is in the driving difficult state in response to: the driver abnormality detection unit detecting the abnormality in the state of the driver; thereafter the operation abnormality detection unit detecting an abnormal operation; and thereafter the traveling abnormality detection unit detecting an abnormal traveling. ### Claim 2 (depends on 1) The state determination device according to claim 1, wherein: in response to detecting the abnormality in the state of the driver, the operation abnormality detection unit starts a detection process for, from among a plurality abnormal operation items predefined for the driving operation, an abnormal operation that is associated with an abnormality content detected by the driver abnormality detection unit. ### Claim 3 (depends on 1) The state determination device according to claim 1, wherein: in response to detecting the abnormality in the driving operation, the traveling abnormality detection unit starts a detection process for, from among a plurality of abnormal traveling items predefined for the traveling state, an abnormal traveling that is associated with the abnormal operation detected by the operation abnormality detection unit. ### Claim 4 (depends on 1) The state determination device according to claim 1, wherein: the integrated determination unit: activates a detection function for the abnormal travel item associated with the abnormal operation detected by the operation abnormality detection unit; or raises an execution priority of the detection function in the traveling abnormality detection unit. ### Claim 5 (depends on 1) The state determination device according to claim 1, wherein: the integrated determination unit: activates a detection function for the abnormal operation item that is associated with the abnormality content detected by the driver abnormality detection unit; or raises an execution priority of the detection function in the operation abnormality detection unit. ### Claim 6 (depends on 1) The state determination device according to claim 1, wherein: the traveling abnormality detection unit acquires a detection signal indicating an abnormal operation output from an abnormality detection device mounted to the vehicle, and uses the detection signal to detect the abnormal operation; and the integrated determination unit acquires a detection signal indicating abnormal traveling output from the abnormality detection device, and uses the detection signal to detect the abnormal traveling. ### Claim 7 (depends on 6) The state determination device according to claim 6, wherein: an operation of the abnormality detection device is stoppable by the driver; and in response to the driver abnormality detection unit detecting the abnormality in the state of the driver, the integrated determination unit activates the abnormality detection device in a stopped state. ### Claim 8 A state determination method performed by at least one processing unit, the method comprising: detecting an abnormality in a state of the driver; detecting an abnormality in a driving operation input by the driver; detecting an abnormality in a traveling state of the vehicle; and determining that the driver is in the driving difficult state in response to: detecting the abnormality in the state of the driver; thereafter detecting an abnormal operation; and thereafter detecting an abnormal traveling, while in response to detecting the abnormality in the state of the driver alone, the method not determining that the driver is in the driving difficult state. ### Claim 9 A computer readable non-transitory storage medium comprising computer-executable instructions that comprise a computer-implemented method of determining that a driver who is driving a vehicle is in a driving difficult state, the method comprising: detecting an abnormality in a state of the driver; detecting an abnormality in a driving operation input by the driver; detecting an abnormality in a traveling state of the vehicle; and determining that the driver is in the driving difficult state in response to: detecting the abnormality in the state of the driver; thereafter detecting an abnormal operation; and thereafter detecting an abnormal traveling, while in response to detecting the abnormality in the state of the driver alone, the method not determining that the driver is in the driving difficult state.