# US 10,503,987 B2 · Apparatus detecting driving incapability state of driver > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,987 B2 - Document ID: US10503987B2 - Status: Granted - Kind: B2 - Application number: 15/320,945 - Filed: 2015-06-08 - Published: 2017-06-08 - Granted: 2019-12-10 - Earliest claimed: 2014-06-23 - Assignees: Denso Corporation - Applicants: Denso Corporation - Inventors: Hiroto Banno; Yutaka Munaoka; Takuhiro Omi; Takeshi Enya - Web page: https://patentgrep.com/patent/US10503987B2 ## CPC - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/1103: {Detecting muscular movement of the eye, e.g. eyelid movement} - A61B 5/1121: {Determining geometric values, e.g. centre of rotation or angular range of movement} - A61B 5/163: {by tracking eye movement, gaze, or pupil change} - A61B 5/18: for vehicle drivers {or machine operators} - A61B 5/6893: {Cars} - B60K 28/06: responsive to incapacity of driver - B60K 35/22: Display screens - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60W 40/02: related to ambient conditions - B60W 40/08: related to drivers or passengers - B60W 40/10: related to vehicle motion - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/23: {Recognition of whole body movements, e.g. for sport training} - G08G 1/0962: having an indicator mounted inside the vehicle, e.g. giving voice messages - A61B 5/1116: {Determining posture transitions} - A61B 2503/22: Motor vehicles operators, e.g. drivers, pilots, captains - A61B 2576/02: specially adapted for a particular organ or body part - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/26: using acoustic output - B60K 2360/1438: Touch screens - B60K 2360/21: using cameras - B60W 2040/0818: {Inactivity or incapacity of driver} - B60W 2050/0088: {Adaptive recalibration} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2540/10: Accelerator pedal position - B60W 2540/12: Brake pedal position - B60W 2540/223: Posture, e.g. hand, foot, or seat position, turned or inclined - B60W 2540/225: Direction of gaze - G01S 19/13: Receivers - G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08) - G16H 30/40: for processing medical images, e.g. editing - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract An apparatus detecting driving incapability state of a driver includes: an inclination detection portion that detects inclination of a head of a driver with respect to a trunk of the driver lower than a neck, based on an image of a driver seat captured by an imaging device mounted on a vehicle; and a posture collapse detection portion that detects the driver is incapable of driving when the inclination of the head detected by the inclination detection portion is greater than a relative inclination threshold during travel of the vehicle. ## Claims ### Claim 1 An apparatus detecting driving incapability state of a driver, the apparatus comprising: an imaging device that is mounted on a vehicle and is configured to capture a driver's seat; a memory storing an inclination detection portion and a posture collapse detection portion; a processor executing the inclination detection portion and the posture collapse detection portion stored in the memory; the inclination detection portion detects inclination of a head of a driver with respect to a trunk of the driver lower than a neck, based on an image of the driver's seat captured by the imaging device; and the posture collapse detection portion detects the driver is incapable of driving when the inclination of the head with respect to the trunk detected by the inclination detection portion is greater than a relative inclination threshold during travel of the vehicle, causing the vehicle to be safely stopped in a case where it is determined that the driver is incapable of driving. ### Claim 2 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, wherein: the inclination detection portion detects center axes of the head and the trunk from the image, and considers inclination of a center axis of the head with respect to a center axis of the trunk as the inclination of the head. ### Claim 3 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, wherein: the inclination detection portion detects the head and a seat belt of the driver seat from the image, and detects the inclination of the head from a positional relation between the seat belt and the head. ### Claim 4 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, wherein: the posture collapse detection portion detects the driver is incapable of driving on condition that a face included in the head does not face in a forward direction of the vehicle for a time longer than a posture collapse determination time. ### Claim 5 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, wherein: the posture collapse detection portion detects the driver is incapable of driving on condition that the inclination of the head is greater than the relative inclination threshold for a time longer than a posture collapse determination time. ### Claim 6 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, wherein: the posture collapse detection portion detects the driver is incapable of driving on condition that a position of the head and a position of the trunk are within an unmoving determination range for a time longer than the posture collapse determination time. ### Claim 7 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, further comprising: a learning portion, stored on the memory and executed by the processor, that learns the inclination of the head detected by the inclination detection portion when the driver is not incapable of driving, wherein: the posture collapse detection portion detects the driver is incapable of driving on condition that the inclination of the head detected by the inclination detection portion is greater than an inclination determination value by exceeding the inclination of the head learned by the learning portion. ### Claim 8 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, wherein: the posture collapse detection portion detects the driver is incapable of driving on condition that a steering wheel of the vehicle is not operated for a time longer than an operation determination time. ### Claim 9 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, further comprising: a shake detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when an amplitude of a shake of the head is smaller than a first amplitude or larger than a second amplitude, which is larger than the first amplitude, by a time a shake determination time elapses after external force has been applied to the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving. ### Claim 10 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, further comprising: a spasm detection portion, stored on the memory and executed by the processor, that detects spasm of the driver, wherein: the relative inclination threshold decreases when the spasm is detected by the spasm detection portion. ### Claim 11 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, wherein: as a vehicle speed of the vehicle is high, the posture collapse determination time is shortened. ### Claim 12 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, wherein: as a time to collision is short, the time being obtained by dividing a vehicle distance from a preceding vehicle by a speed relative to the preceding vehicle, the posture collapse determination time is shortened. ### Claim 13 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, further comprising: a storage portion in which personal information on the driver, including medical history and age, is registered, wherein: the posture collapse determination time is set based on the personal information registered in the storage portion. ### Claim 14 (depends on 5) The apparatus detecting driving incapability state of the driver according to claim 5, wherein: the posture collapse determination time is prolonged when a drive assist control is executed in the vehicle. ### Claim 15 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, further comprising: a posture notification portion that notifies the driver of a degree of posture collapse of the driver based on the inclination of the head detected by the inclination detection portion. ### Claim 16 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, wherein: the driving incapability state includes a state in which the driver has a sudden illness. ### Claim 17 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, further comprising: a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold for a time longer than a direction collapse determination time, during travel of the vehicle in a case where the posture collapse detection portion does not detect the driver is incapable of driving. ### Claim 18 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, further comprising: a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold and the driver releases a steering wheel of the vehicle, during travel of the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving. ### Claim 19 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, further comprising: a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold and an opening degree of an accelerator is greater than a predetermined opening degree, during travel of the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving. ### Claim 20 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, further comprising: a face direction detection portion, stored on the memory and executed by the processor, that detects a direction of the face of the driver with respect to a forward direction of the vehicle based on the image of the driver seat captured by the imaging device mounted on the vehicle; and a direction collapse detection portion, stored on the memory and executed by the processor, that detects the driver is incapable of driving when the direction of the face detected by the face direction detection portion is greater than a face direction threshold and an accelerator operation and a brake operation are not performed for a time longer than an operation determination time, during travel of the vehicle, in a case where the posture collapse detection portion does not detect the driver is incapable of driving. ### Claim 21 (depends on 18) The apparatus detecting driving incapability state of the driver according to claim 18, wherein: the direction collapse detection portion detects the driver is incapable of driving on condition that the driver releases the steering wheel for a time longer than an operation determination time. ### Claim 22 (depends on 19) The apparatus detecting driving incapability state of the driver according to claim 19, wherein: the direction collapse detection portion detects the driver is incapable of driving on condition that the opening degree of the accelerator is larger than the predetermined opening degree for a time longer than an operation determination time. ### Claim 23 (depends on 19) The apparatus detecting driving incapability state of the driver according to claim 19, wherein: the direction collapse detection portion detects the driver is incapable of driving on condition that the direction of the face is greater than the face direction threshold for a time longer than a direction collapse determination time. ### Claim 24 (depends on 23) The apparatus detecting driving incapability state of the driver according to claim 23, wherein: the direction collapse detection portion detects the driver is incapable of driving when the face faces downward greatly exceeding a downward threshold or upward greatly exceeding an upward threshold. ### Claim 25 (depends on 18) The apparatus detecting driving incapability state of a driver according to claim 18, wherein: the direction collapse detection portion detects the driver is incapable of driving on condition that a hand of the driver is lower than a neck of the driver. ### Claim 26 (depends on 18) The apparatus detecting driving incapability state of the driver according to claim 18, further comprising: a learning portion, stored on the memory and executed by the processor, that learns the direction of the face detected by the face direction detection portion when the driver is not incapable of driving, wherein: the direction collapse detection portion detects the driver is incapable of driving on condition that the direction of the face detected by the face direction detection portion is greater than the direction of the face learned by the learning portion by exceeding an inclination determination value. ### Claim 27 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, wherein: the inclination detection portion detects, on a basis of a neck position of the driver, the inclination of the head with respect to the trunk based on the image of the driver seat; and the posture collapse detection portion detects the driver is incapable of driving in response to the inclination being greater than a relative inclination threshold. ### Claim 28 (depends on 1) The apparatus detecting driving incapability state of the driver according to claim 1, wherein the inclination detection portion detects, on a basis of a neck position of the driver, an inclination angle between the head and the trunk based on the image of the driver seat, as the inclination of the head; and the posture collapse detection portion determines that the driver is in a driving incapable state in response to a determination that the inclination angle is greater than a predetermined threshold. ### Claim 29 An apparatus detecting driving incapability state of a driver of a vehicle, the apparatus comprising: a camera configured to capture a driver's seat of the vehicle; and a processor configured to detect inclination of a head of the driver with respect to a trunk of the driver lower than a neck of the driver, based on an image of the driver's seat captured by the camera, and detect the driver is incapable of driving when the detected inclination of the head with respect to the trunk is greater than a relative inclination threshold during travel of the vehicle, causing the vehicle to be safely stopped in a case where it is determined that the driver is incapable of driving. ### Claim 30 A method of detecting driving incapability state of a driver of a vehicle, the method comprising: capturing, by a camera, a driver's seat of the vehicle; detecting, by a processor, inclination of a head of the driver with respect to a trunk of the driver lower than a neck of the driver, based on an image of the driver's seat captured by the camera; and detecting, by the processor, the driver is incapable of driving when the detected inclination of the head with respect to the trunk is greater than a relative inclination threshold during travel of the vehicle, causing the vehicle to be safely stopped in a case where it is determined that the driver is incapable of driving.