# US 2021/0276569 A1 · Driver condition estimating device, driver condition estimating method and computer program therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0276569 A1
- Document ID: US20210276569A1
- Status: Published application
- Kind: A1
- Application number: 17/104,007
- Filed: 2020-11-25
- Published: 2021-09-09
- Earliest claimed: 2020-03-06
- Assignees: Mazda Motor Corporation
- Applicants: Mazda Motor Corporation
- Inventors: Makoto YOSHIDA; Taro IWAMOTO; Ken IKUHISA; Yohei IWASHITA
- Web page: https://patentgrep.com/patent/US20210276569A1

## CPC

- B60W 40/08: related to drivers or passengers
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- B60W 2050/146: {Display means}
- B60W 2540/21: Voice
- B60W 2540/221: Physiology, e.g. weight, heartbeat, health or special needs
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A driver condition estimating device includes circuitry configured to measure movement of the head of a driver output from a driver camera that photographs the driver and detect a sign of abnormality of the driver from the movement of the head. The circuitry determines existence of the sign of abnormality of the driver by calculating a periodic feature amount from time series data showing movement of the head of the driver, calculating coherence between the movement of the head of the driver and lateral acceleration acting on the head of the driver, calculating time series variation patterns from the obtained periodic feature amount and the obtained coherence, and comparing the obtained time series variation patterns with a predetermined threshold.

## Claims

### Claim 1

A driver condition estimating device configured to estimate condition of a driver who is driving a vehicle, the driver condition estimating device comprising:
circuitry configured to:
measure movement of a head of the driver from output of a camera that photographs the driver; and
detect a sign of abnormality of the driver from the movement of the head;
calculate a periodic feature amount from time series data showing the movement of the head of the driver;
calculate coherence between the movement of the head of the driver and lateral acceleration acting on the head of the driver;
calculate time series variation patterns from the calculated periodic feature amount and the calculated coherence; and
compare the calculated time series variation patterns with a predetermined threshold to determine existence of the sign of abnormality of the driver.

### Claim 2 (depends on 1)

The driver condition estimating device according to claim 1, wherein the detector is configured to obtain lateral acceleration acting on the head of the driver, from output of a sensor for measuring a movement state of the vehicle.

### Claim 3 (depends on 2)

The driver condition estimating device according to claim 2, wherein the circuitry is configured to:
recognize a head of the driver in a photographic image obtained by the camera, and
measure movement of the head of the driver from the photographic image, wherein
the movement of the head of the driver includes angles of the head.

### Claim 4 (depends on 2)

The driver condition estimating device according to claim 2, wherein the circuitry is configured to:
calculate periodic feature amounts from the time series data of movement of the head, and
calculate autocorrelation indexes as the periodic feature amounts by detrended fluctuation analysis.

### Claim 5 (depends on 2)

The driver condition estimating device according to claim 2, wherein the circuitry is configured to:
output an inquiry to the driver in response to the abnormality being determined, wherein the inquiry is at least one of a vocal inquiry output by a speaker in the vehicle and a visual inquiry shown on a display in the vehicle.

### Claim 6 (depends on 2)

The driver condition estimating device according to claim 2, wherein
the detector is configured to:
dimensionally reduce time series data of the calculated periodic feature amount and of the calculated coherence by using a nonlinear dimensionality reduction method to obtain two-dimensional data as the time series variation pattern; and
determine existence of the sign of abnormality of the driver from the obtained two-dimensional data by using a determination line in a two-dimensional map as the predetermined threshold.

### Claim 7 (depends on 6)

The driver condition estimating device according to claim 6, wherein the circuitry is configured to:
recognize a head of the driver in a photographic image obtained by the camera, and
measure movement of the head of the driver from the photographic image, wherein
the movement of the head of the driver includes angles of the head.

### Claim 8 (depends on 6)

The driver condition estimating device according to claim 6, wherein the circuitry is configured to:
calculate periodic feature amounts from the time series data of movement of the head, and
calculate autocorrelation indexes as the periodic feature amounts by detrended fluctuation analysis.

### Claim 9 (depends on 6)

The driver condition estimating device according to claim 6, wherein the circuitry is configured to:
output an inquiry to the driver in response to the abnormality being determined, wherein the inquiry is at least one of a vocal inquiry output by a speaker in the vehicle and a visual inquiry shown on a display in the vehicle.

### Claim 10 (depends on 1)

The driver condition estimating device according to claim 1, wherein
the detector is configured to:
dimensionally reduce time series data of the calculated periodic feature amount and of the calculated coherence by using a nonlinear dimensionality reduction method to obtain two-dimensional data as the time series variation pattern; and
determine existence of the sign of abnormality of the driver from the obtained two-dimensional data by using a determination line in a two-dimensional map as the predetermined threshold.

### Claim 11 (depends on 1)

The driver condition estimating device according to claim 1, wherein the circuitry is configured to:
recognize a head of the driver in a photographic image obtained by the camera, and
measure movement of the head of the driver from the photographic image, wherein
the movement of the head of the driver includes angles of the head.

### Claim 12 (depends on 11)

The driver condition estimating device according to claim 11, wherein the circuitry is configured to:
calculate periodic feature amounts from the time series data of movement of the head, and
calculate autocorrelation indexes as the periodic feature amounts by detrended fluctuation analysis.

### Claim 13 (depends on 12)

The driver condition estimating device according to claim 12, wherein the circuitry is configured to:
output an inquiry to the driver in response to the abnormality being determined, wherein the inquiry is at least one of a vocal inquiry output by a speaker in the vehicle and a visual inquiry shown on a display in the vehicle.

### Claim 14 (depends on 11)

The driver condition estimating device according to claim 11, wherein the circuitry is configured to:
output an inquiry to the driver in response to the abnormality being determined, wherein the inquiry is at least one of a vocal inquiry output by a speaker in the vehicle and a visual inquiry shown on a display in the vehicle.

### Claim 15 (depends on 1)

The driver condition estimating device according to claim 1, wherein the circuitry is configured to:
calculate periodic feature amounts from the time series data of movement of the head, and
calculate autocorrelation indexes as the periodic feature amounts by detrended fluctuation analysis.

### Claim 16 (depends on 1)

The driver condition estimating device according to claim 1, wherein the circuitry is configured to:
output an inquiry to the driver in response to the abnormality being determined, wherein the inquiry is at least one of a vocal inquiry output by a speaker in the vehicle and a visual inquiry shown on a display in the vehicle.

### Claim 17

A driver condition estimating method for estimating condition of a driver who is driving a vehicle, the driver condition estimating method comprising:
calculating a periodic feature amount from time series data showing movement of a head of the driver;
calculating coherence between the movement of the head of the driver and lateral acceleration acting on the head of the driver;
calculating time series variation patterns from the calculated periodic feature amount and the calculated coherence; and
comparing the calculated time series variation patterns with a predetermined threshold to determine existence of a sign of abnormality of the driver.

### Claim 18

A non-transitory computer readable storage including computer readable instructions that when executed by a controller cause the controller to execute a driver state determination method for a driver in a vehicle, the method comprising:
calculating a periodic feature amount from time series data showing movement of a head of the driver;
calculating coherence between the movement of the head of the driver and lateral acceleration acting on the head of the driver;
calculating time series variation patterns from the calculated periodic feature amount and the calculated coherence; and
comparing the calculated time series variation patterns with a predetermined threshold to determine existence of a sign of abnormality of the driver.
