# US 11,077,858 B2 · Vehicle driver state determination apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,077,858 B2
- Document ID: US11077858B2
- Status: Granted
- Kind: B2
- Application number: 16/431,985
- Filed: 2019-06-05
- Published: 2019-09-19
- Granted: 2021-08-03
- Earliest claimed: 2016-12-22
- Assignees: Denso Corporation
- Applicants: Denso Corporation
- Inventors: Hiroshi Morimoto; Ayako Tsunoda; Kaneyoshi Ueda
- Web page: https://patentgrep.com/patent/US11077858B2

## CPC

- B60W 40/09: Driving style or behaviour
- B60R 21/00: Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks (safety belts or body harnesses in vehicles B60R22/00; seats constructed to protect the occupant from the effect of abnormal g-forces, e.g. crash or safety seats, B60N2/42; energy-absorbing arrangements for hand wheels for steering vehicles B62D1/11; energy-absorbing arrangements for vehicle steering columns B62D1/19)
- B60W 30/08: {Active safety systems} predicting or avoiding probable or impending collision {or attempting to minimise its consequences}
- B60W 30/17: with provision for special action when the preceding vehicle comes to a halt, e.g. stop and go
- B60W 40/08: related to drivers or passengers
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- B60W 50/16: Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/168: {Feature extraction; Face representation}
- G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08)
- B60W 2040/0818: {Inactivity or incapacity of driver}
- B60W 2050/143: {Alarm means}
- B60W 2050/146: {Display means}
- B60W 2540/225: Direction of gaze
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A vehicle driver state determination apparatus in a vehicle is provided as follows. A specific notification is issued from a driving assistance apparatus in response to an occurrence of a specific event; the specific notification urges the driver to perform a predetermined driving operation. A reaction time is measured from when the specific notification is issued to when the predetermined driving operation is performed by the driver, to evaluate a state relating to a driving ability of the driver.

## Claims

### Claim 1

A vehicle driver state determination apparatus provided in a vehicle including a driving assistance apparatus, the vehicle driver state determination apparatus determining a state of a driver of the vehicle, comprising:
a determination device configured to determine that a specific notification is issued to the driver from the driving assistance apparatus in response to an occurrence of a specific event, the specific notification urging the driver to perform a predetermined driving operation;
a time measurement device configured to measure a reaction time from when the specific notification is issued to when the predetermined driving operation is performed by the driver, to provide a time measurement result; and
an evaluation device configured to evaluate a state relating to a driving ability of the driver from the time measurement result.

### Claim 2 (depends on 1)

The vehicle driver state determination apparatus according to claim 1, further comprising:
a direction detection unit configured to detect either a sight-line direction of the driver or a face direction of the driver, wherein
in response to that the determination device determines that the specific notification is issued under a condition that either the sight-line direction of the driver or the face direction of the driver detected by the direction detection unit fails to face an area in front of the vehicle, the reaction time is measured by the time measurement device and the state is evaluated by the evaluation device.

### Claim 3 (depends on 1)

The vehicle driver state determination apparatus according to claim 1, wherein:
the driving assistance apparatus includes an adaptive cruise control function;
the determination device determines that a preceding vehicle start notification is issued as the specific notification from the driving assistance apparatus that is under an execution of the adaptive cruise control function, the preceding vehicle start notification indicating that a preceding vehicle has started after the preceding vehicle stopped and the vehicle then stopped by following the preceding vehicle;
the time measurement device measures the reaction time to when a starting operation of starting the vehicle is performed as the predetermined driving operation by the driver; and
in response to that the time measurement result of the reaction time is equal to or greater than a threshold value, the evaluation device determines a reaction delay of the driver.

### Claim 4 (depends on 1)

The vehicle driver state determination apparatus according to claim 1, wherein:
the driving assistance apparatus includes a function of detecting a pedestrian running out and issuing a pedestrian running out notification indicating that a pedestrian runs out;
the determination device determines that the pedestrian running out notification is issued as the specific notification;
the time measurement device measures the reaction time to when a decelerating operation of decelerating the vehicle is performed as the predetermined driving operation by the driver; and
in response to that the time measurement result of the reaction time is equal to or greater than a threshold value, the evaluation device determines a reaction delay of the driver.

### Claim 5 (depends on 1)

The vehicle driver state determination apparatus according to claim 1, wherein:
the driving assistance apparatus includes a function of issuing a preceding vehicle start notification indicating a start of a preceding vehicle;
the determination device determines that the preceding vehicle start notification is issued as the specific notification;
the time measurement device measures a reaction time to when a brake off operation or an accelerator on operation is performed as the predetermined driving operation by the driver; and
in response to that the time measurement result of the reaction time is equal to or greater than a threshold value, the evaluation device determines a reaction delay of the driver.

### Claim 6 (depends on 1)

The vehicle driver state determination apparatus according to claim 1, wherein:
the driving assistance apparatus includes a function of issuing a notification of a forward traffic signal changing to a green signal;
the determination device determines that the notification of the forward traffic signal changing to the green light is issued as the specific notification;
the time measurement device measures a reaction time from when the notification of the forward traffic signal changing to the green light is issued to when a brake off operation or an accelerator on operation is performed as the predetermined driving operation by the driver; and
in response to that the time measurement result of the reaction time is equal to or greater than a threshold value, the evaluation device determines a reaction delay of the driver.

### Claim 7 (depends on 1)

The vehicle driver state determination apparatus according to claim 1, wherein
in response to that the predetermined driving operation has not been performed by the driver despite a predetermined period of time having elapsed since the occurrence of the specific event, the evaluation device determines that the driver has an intention not to perform a driving operation.

### Claim 8 (depends on 1)

The vehicle driver state determination apparatus according to claim 1, wherein:
the time measurement is further configured to provide the measured reaction time as the time measurement result to evaluate the state relating to the driving ability of the driver; and
the evaluation device is further configured
to determine whether the measured reaction time is equal to or greater than a threshold value, and
to alert the driver that the driver is in a state where the driving ability of the driver is reduced, in response to determining that the measured reaction time is equal to or greater than the threshold value.

### Claim 9

A vehicle driver state determination apparatus provided in a vehicle, comprising:
at least one control circuit connected with a driving assistance apparatus via a communication link, the at least one control circuit being configured
to determine that a specific notification is issued to the driver from the driving assistance apparatus at least visually or audibly or tactually by using an in-vehicle output device in response to an occurrence of a specific event during driving of the vehicle, the specific notification urging the driver to perform a predetermined driving operation to drive the vehicle,
to measure, in response to determining that the specific notification is issued, a reaction time from when the specific notification is issued to when the predetermined driving operation is performed by the driver to operate either a brake or an accelerator,
to provide the measured reaction time as a time measurement result to evaluate a state relating to a driving ability of the driver,
to determine whether the measured reaction time is equal to or greater than a threshold value, and
to alert the driver that the driver is in a state where the driving ability of the driver is reduced, in response to determining that the measured reaction time is equal to or greater than the threshold value.

### Claim 10 (depends on 9)

The vehicle driver state determination apparatus according to claim 9, further comprising:
at least one different control circuit connected with the at least one control circuit via a communication link, the at least one different control circuit including a processor processing an image of the driver captured by a camera communicating with the processor, the processor being configured to detect a direction of a driver face of the driver that is either a sight-line direction of the driver or a face direction of the driver based on the processed image, the at least one different control circuit providing a result of the detected direction of the driver face, wherein
in response to that the specific notification is determined to be issued under a condition that the direction of the driver face is determined to fail to face an area in front of the vehicle based on the result of the detected direction of the driver face from the at least one different control circuit, the at least one control circuit measures the reaction time to evaluate the state relating to the driving ability of the driver.

### Claim 11 (depends on 9)

The vehicle driver state determination apparatus according to claim 9, wherein:
the in-vehicle output device includes at least one of a display or a speaker or a vibrator.

### Claim 12 (depends on 9)

The vehicle driver state determination apparatus according to claim 9, wherein:
the driving assistance apparatus includes an adaptive cruise control function; and
the at least one control circuit is further configured
to determine that a preceding vehicle start notification is issued as the specific notification from the driving assistance apparatus that is under an execution of the adaptive cruise control function, the preceding vehicle start notification indicating that a preceding vehicle has started after the preceding vehicle stopped and the vehicle then stopped by following the preceding vehicle,
to measure the reaction time to when a starting operation of starting the vehicle is performed as the predetermined driving operation by the driver, and
to determine a reaction delay of the driver in response to the time measurement result of the reaction time being equal to or greater than a threshold value.

### Claim 13 (depends on 9)

The vehicle driver state determination apparatus according to claim 9, wherein:
the driving assistance apparatus includes a function of detecting a pedestrian running out and issuing a pedestrian running out notification indicating that a pedestrian runs out; and
the at least one control circuit is further configured
to determine that the pedestrian running out notification is issued as the specific notification,
to measure the reaction time to when a decelerating operation of decelerating the vehicle is performed as the predetermined driving operation by the driver, and
to determine a reaction delay of the driver in response to the time measurement result of the reaction time being equal to or greater than a threshold value.

### Claim 14 (depends on 9)

The vehicle driver state determination apparatus according to claim 9, wherein:
the driving assistance apparatus includes a function of issuing a preceding vehicle start notification indicating a start of a preceding vehicle; and
the at least one control circuit is further configured
to determine that the preceding vehicle start notification is issued as the specific notification,
to measure a reaction time to when a brake off operation or an accelerator on operation is performed as the predetermined driving operation by the driver, and
to determine a reaction delay of the driver in response to the time measurement result of the reaction time being equal to or greater than a threshold value.

### Claim 15 (depends on 9)

The vehicle driver state determination apparatus according to claim 9, wherein:
the driving assistance apparatus includes a function of issuing a notification of a forward traffic signal changing to a green signal; and
the at least one control circuit is further configured
to determine that the notification of the forward traffic signal changing to the green light is issued as the specific notification,
to measure a reaction time from when the notification of the forward traffic signal changing to the green light is issued to when a brake off operation or an accelerator on operation is performed as the predetermined driving operation by the driver, and
to determine a reaction delay of the driver.

### Claim 16 (depends on 9)

The vehicle driver state determination apparatus according to claim 9, wherein
in response to the predetermined driving operation having not been performed by the driver despite a predetermined period of time having elapsed since the occurrence of the specific event, the at least one control circuit is further configured to determine that the driver has an intention not to perform a driving operation.

### Claim 17

A vehicle driver state determination apparatus provided in a vehicle, comprising:
at least one control circuit connected with a driving assistance apparatus via a communication link, the at least one control circuit being configured
to determine that a specific notification is issued to the driver from the driving assistance apparatus at least visually, audibly, or tactually by using an in-vehicle output device, in response to an occurrence of a specific event, the specific notification urging the driver to perform a predetermined driving operation,
to measure, in response to determining that the specific notification is issued, a reaction time from when the specific notification is issued to when the predetermined driving operation is performed by the driver to operate either a brake or an accelerator, and
to provide a time measurement result to evaluate a state relating to a driving ability of the driver, wherein:
the at least one control circuit is further configured
to determine that a preceding vehicle start notification is issued as the specific notification from the driving assistance apparatus that is under an execution of the adaptive cruise control function, the preceding vehicle start notification indicating that a preceding vehicle has started after the preceding vehicle stopped and the vehicle then stopped by following the preceding vehicle,
to measure the reaction time to when a starting operation of starting the vehicle is performed as the predetermined driving operation by the driver, and
to determine a reaction delay of the driver in response to the time measurement result of the reaction time being equal to or greater than a threshold value.