# US 2018/0022358 A1 · System and method for responding to driver state

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0022358 A1
- Document ID: US20180022358A1
- Status: Published application
- Kind: A1
- Application number: 15/720,489
- Filed: 2017-09-29
- Published: 2018-01-25
- Earliest claimed: 2013-03-15
- Assignees: Honda Motor Co., Ltd.
- Applicants: Honda Motor Co., Ltd.
- Inventors: Kin C. Fung; Timothy J. Dick
- Web page: https://patentgrep.com/patent/US20180022358A1

## CPC

- G06V 10/82: using neural networks
- B60R 25/25: using biometry
- B60W 40/08: related to drivers or passengers
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/45: {Detection of the body part being alive}
- G06V 40/70: Multimodal biometrics, e.g. combining information from different biometric modalities
- G07C 5/02: Registering or indicating driving, working, idle, or waiting time only (apparatus forming part of taximeters G07B13/00)
- G07C 5/08: Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C 9/37: using biometric data, e.g. fingerprints, iris scans or voice recognition
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- B60W 2540/22: Psychological state; Stress level or workload
- B60W 2540/221: Physiology, e.g. weight, heartbeat, health or special needs
- B60W 2540/223: Posture, e.g. hand, foot, or seat position, turned or inclined
- B60W 2540/225: Direction of gaze
- B60W 2540/229: Attention level, e.g. attentive to driving, reading or sleeping
- G06F 2218/12: Classification; Matching
- G06V 40/15: {Biometric patterns based on physiological signals, e.g. heartbeat, blood flow}
- G07C 9/00309: {operated with bidirectional data transmission between data carrier and locks}
- G07C 9/00563: {using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for controlling vehicle systems includes receiving monitoring information from one or more monitoring systems and determining a plurality of driver states based on the monitoring information from the one or more monitoring systems. The method includes determining a combined driver state based on the plurality of driver states and modifying control of one or more vehicle systems based on the combined driver state.

## Claims

### Claim at position 1

-65. (canceled)

### Claim 66

A method of controlling a vehicle system in a vehicle, comprising:
receiving monitoring information from one or more monitoring systems of the vehicle;
determining a first driver state of a driver and a second driver state of the driver based on the monitoring information from the one or more monitoring systems, wherein the first driver state is a behavioral driver state of the driver and the second driver state is a physiological driver state of the driver;
determining a combined driver state index by evaluating the first driver state and the second driver state together;
modifying a system status of the vehicle system as a function of the combined driver state index to increase alertness of the driver;
determining a third driver state of the driver based on the monitoring information from the one or more monitoring systems, wherein the third driver state is a vehicular sensed driver state;
determining a change in the combined driver state index based on the third driver state; and
modifying the vehicle system as a function of the third driver state.

### Claim 67 (depends on 66)

The method of claim 66, wherein the first driver state is determined based on eye movement information received from the one or more monitoring systems, and the second driver state is determined based on heart rate information received from the one or more monitoring systems.

### Claim 68 (depends on 66)

The method of claim 66, wherein the one or more monitoring systems includes a portable device associated with the driver, wherein the portable device stores user data about the driver tracked over time.

### Claim 69 (depends on 68)

The method of claim 68, wherein the second driver state is based on the user data about the driver tracked over time.

### Claim 70 (depends on 66)

The method of claim 66, including determining a vehicle state of the vehicle based on the monitoring information from the one or more monitoring systems, wherein the vehicle state is the system status of the vehicle system.

### Claim 71 (depends on 70)

The method of claim 70, including upon determining the driver is distracted based on the combined driver state index and upon determining the vehicle system is ON based on the system status, modifying the system status of the vehicle system as a function of the combined driver state index to increase alertness of the driver includes modifying the system status of the vehicle system to OFF.

### Claim 72 (depends on 66)

The method of claim 66, wherein determining the change in the combined driver state index based on the third driver state includes determining an increase in alertness of the driver from the combined driver state index.

### Claim 73 (depends on 66)

The method of claim 66, wherein the third driver state is an information transfer rate between the driver and the vehicle.

### Claim 74 (depends on 73)

The method of claim 73, wherein determining the change in the combined driver state index based on the third driver state includes determining an increase in alertness of the driver based on determining a high information transfer rate between the driver and the vehicle.

### Claim 75 (depends on 66)

The method of claim 66, wherein upon determining the driver is alert based on the change in the combined driver state index, modifying the vehicle system as a function of the third driver state includes modifying the system status to ON.

### Claim 76 (depends on 66)

The method of claim 66, wherein upon determining the driver is distracted based on the change in the combined driver state index, modifying the vehicle system as a function of the third driver state includes modifying the system status of the vehicle system to OFF for a period of time.

### Claim 77 (depends on 76)

The method of claim 76, determining a fourth driver state of the driver, wherein the fourth driver state is a physiological driver state of the driver, and the fourth driver state is based on stored monitoring information about the driver tracked over time received from a portable device of the driver.

### Claim 78 (depends on 77)

The method of claim 77, including determining the driver is alert based on the fourth driver state and modifying the system status of the vehicle system to ON.

### Claim 79

A method of controlling a vehicle system in a vehicle, comprising:
receiving monitoring information from one or more monitoring systems of the vehicle;
determining a first driver state of a driver and a second driver state of the driver based on the monitoring information from the one or more monitoring systems;
determining a combined driver state index by evaluating the first driver state and the second driver state together;
modifying a system status of the vehicle system as a function of the combined driver state index to increase attention of the driver;
determining a third driver state of the driver based on the monitoring information from the one or more monitoring systems;
determining a second combined driver state index based on the third driver state; and
modifying the vehicle system by evaluating the combined driver state index and the second combined driver state index together.

### Claim 80 (depends on 79)

The method of claim 79, wherein the first driver state is a behavioral driver state based on behavioral information received from the one or more monitoring systems, the second driver state is a physiological driver state based on physiological information received from the one or more monitoring systems, and the third driver state is a vehicular sensed driver state based on vehicle information from the one or more monitoring systems.

### Claim 81 (depends on 79)

The method of claim 79, including identifying the driver based on the monitoring information from the one or more monitoring systems and receiving stored user data about the driver tracked over time from a wearable device associated with the driver.

### Claim 82 (depends on 81)

The method of claim 81, wherein the second driver state is based on the stored user data.

### Claim 83 (depends on 79)

The method of claim 79, including determining a vehicle state of the vehicle based on the monitoring information from the one or more monitoring systems, wherein the vehicle state is an auto control mode status indicating whether the vehicle is enabled for operating under automatic control.

### Claim 84 (depends on 83)

The method of claim 83, including upon determining the auto control mode status is ON and upon determining the combined driver state index indicates an inattentive driver, modifying the system status of the vehicle system as a function of the combined driver state index to increase attention of the driver includes modifying the system status to OFF.

### Claim 85 (depends on 79)

The method of claim 79, wherein modifying the system status of the vehicle system as a function of the combined driver state index to increase attention of the driver includes modifying the system status of the vehicle system to increase active driving of the vehicle by the driver.

### Claim 86 (depends on 85)

The method of claim 85, wherein determining the third driver state of the driver is based on vehicle information from the one or more monitoring systems, wherein the vehicle information is an information transfer rate between the driver and the vehicle in response to the driver transmitting information to the vehicle during active driving of the vehicle by the driver.

### Claim 87 (depends on 79)

The method of claim 79, wherein modifying the system status of the vehicle system as a function of the combined driver state index to increase attention of the driver includes modifying the system status of the vehicle system to increase manual control of the vehicle by the driver.

### Claim 88 (depends on 79)

The method of claim 79, wherein modifying the system status of the vehicle system as a function of the combined driver state index to increase attention of the driver includes modifying the system status of the vehicle system to OFF, and upon determining an increase in attention based on evaluating the combined driver state index and the second driver state together, modifying the vehicle system includes reactivating the vehicle system.

### Claim 89 (depends on 79)

The method of claim 79, including determining a fourth driver state of the driver based on the monitoring information from the one or more monitoring systems, wherein the fourth driver state is a behavioral driver state, and the fourth driver state is determined based on eye movement information received from the one or more monitoring systems.

### Claim 90 (depends on 89)

The method of claim 89, wherein determining the second combined driver state index based on the third driver state includes determining the second combined driver state index based on the third driver state and the fourth driver state.

### Claim 91 (depends on 79)

The method of claim 79, wherein modifying the vehicle system by evaluating the combined driver state index and the second combined driver state index together includes determining a change in the combined driver state index.

### Claim 92 (depends on 91)

The method of claim 91, wherein determining the change in the combined driver state index includes determining an increase in attention of the driver by comparing the combined driver state index to the second combined driver state index.

### Claim 93 (depends on 79)

The method of claim 79, including upon determining the attention of the driver meets a predetermined level based evaluating the combined driver state index and the second combined driver state index together, modifying the vehicle includes modifying the system status of the vehicle system to ON.

### Claim 94 (depends on 79)

The method of claim 79, including upon determining the driver is distracted by evaluating the combined driver state index and the second combined driver state index together, modifying the vehicle system includes disabling the vehicle system for a period of time.

### Claim 95

A system for controlling a vehicle system in a vehicle, comprising:
a monitoring system for receiving monitoring information; and
an electronic control unit including a processor, wherein the processor:
determines a first driver state of a driver and a second driver state of the driver based on the monitoring information from the monitoring system, wherein the first driver state is a behavioral driver state of the driver and the second driver state is a physiological driver state of the driver;
determines a combined driver state index by evaluating the first driver state and the second driver state together;
modifies a system status of the vehicle system based on the combined driver state index to improve alertness of the driver;
determines a third driver state of the driver based on the monitoring information from the monitoring system, wherein the third driver state is a vehicular sensed driver state;
determines a change in the combined driver state index based on the third driver state; and
modifies the vehicle system as a function of the third driver state.

### Claim 96 (depends on 95)

The system of claim 95, wherein the monitoring system includes a wearable device associated with the driver, and wherein the wearable device tracks and stores user data patterns about the driver over time.

### Claim 97 (depends on 96)

The system of claim 96, wherein the processor determines the second driver state based on the user data patterns.

### Claim 98 (depends on 95)

The system of claim 95, wherein the processor modifies the system status of the vehicle system as a function of the combined driver state index by setting the system status of the vehicle system to require an increase in active driving of the vehicle by the driver to improve alertness of the driver.

### Claim 99 (depends on 95)

The system of claim 95, wherein the monitoring information indicates a flow of information from the driver to the vehicle, the processor calculates an information transfer rate based on the flow of information from the driver to the vehicle, and the processor determines the third driver state of the driver based on the information transfer rate.

### Claim 100 (depends on 95)

The system of claim 95, wherein upon the processor determining the driver is alert based on the change in the combined driver state index, the processor modifies the system status of the vehicle system to ON.

### Claim 101 (depends on 95)

The system of claim 95, wherein upon the processor determining the driver is distracted based on the change in the combined driver state index, the processor modifies the system status of the vehicle system to OFF for a period of time.

### Claim 102 (depends on 96)

The system of claim 96, wherein the processor determines a fourth driver state of the driver, wherein the fourth driver state is a physiological driver state of the driver, and the fourth driver state is based on the user data patterns received from the wearable device.

### Claim 103 (depends on 102)

The system of claim 102, wherein the processor determines the driver is alert based on the fourth driver state and the processor modifies the system status of the vehicle system to ON based on the fourth driver state.
