# US 2022/0067411 A1 · Inattentiveness determination device, inattentiveness determination system, inattentiveness determination method, and storage medium for storing program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0067411 A1
- Document ID: US20220067411A1
- Status: Published application
- Kind: A1
- Application number: 17/415,240
- Filed: 2019-12-24
- Published: 2022-03-03
- Earliest claimed: 2018-12-27
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Nana JUMONJI
- Web page: https://patentgrep.com/patent/US20220067411A1

## CPC

- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/18: for vehicle drivers {or machine operators}
- A61B 5/6893: {Cars}
- G06V 40/18: Eye characteristics, e.g. of the iris
- G08B 21/06: indicating a condition of sleep, e.g. anti-dozing alarms
- A61B 2503/22: Motor vehicles operators, e.g. drivers, pilots, captains
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- G08G 3/02: Anti-collision systems
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

An inattentiveness determination device includes a photographed image acquisition means configured to acquire a photographed image of a driver driving a moving object, a first inattentiveness determination means which detects a gaze direction of the driver according to the photographed image, and determine whether or not the driver is in an inattentiveness state on a basis of a predetermined gaze range based on a moving direction of the moving object and the gaze direction, and a second inattentiveness determination means configured to, if the gaze direction cannot be detected, determine whether or not the driver is in the inattentiveness state on a basis of a face direction of the driver and a predetermined range of the face direction based on the moving direction of the moving object.

## Claims

### Claim 1

An inattentiveness determination device, comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to;
acquire a photographed image of a driver driving a moving object;
detect a gaze direction of the driver according to the photographed image, and determine whether or not the driver is in an inattentiveness state on a basis of a predetermined gaze range based on a moving direction of the moving object and the gaze direction; and
determine, if the gaze direction cannot be detected, whether or not the driver is in an inattentiveness state on a basis of a face direction of the driver and a predetermined range of the face direction based on the moving direction of the moving object.

### Claim 2 (depends on 1)

The inattentiveness determination device according to claim 1,
wherein the at least one processor is configured to execute the instructions to:
measure a time when the gaze direction is not in the predetermined gaze range and determine the inattentiveness state if a ratio per unit time of the measured time is equal to or greater than a first predetermined ratio.

### Claim 3 (depends on 1)

The inattentiveness determination device according to claim 1 wherein the at least one processor is configured to execute the instructions to:
measure a time when the face direction is not in the predetermined range of the face direction and determine the inattentiveness state if a ratio per unit time of the measured time is equal to or greater than a second predetermined ratio.

### Claim 4 (depends on 1)

The inattentiveness determination device according to claim 1,
wherein the at least one processor is further configured to execute the instructions to:
determine, in a case where the gaze direction and the face direction cannot be detected, whether or not the driver's face can be detected on a basis of the photographed image and determine the inattentiveness state if the driver's face cannot be detected.

### Claim 5 (depends on 4)

The inattentiveness determination device according to claim 4,
wherein the at least one processor is configured to execute the instructions to:
detect a time when the driver's face cannot be detected, and determine the inattentiveness state when a ratio per unit time of the detected time is equal to or greater than a third predetermined ratio.

### Claim 6

(canceled)

### Claim 7

An inattentiveness determination method comprising:
acquiring a photographed image of a driver driving a moving object;
detecting a gaze direction of the driver according to the photographed image, and determining whether or not the driver is in an inattentiveness state on a basis of a predetermined gaze range based on a moving direction of the moving object and the gaze direction; and
determining, if the gaze direction cannot be detected, whether or not the driver is in the inattentiveness state on a basis of a face direction of the driver and a predetermined range of the face direction based on the moving direction of the moving object.

### Claim 8

A non-transitory computer-readable storage medium that stores a program that executes in an inattentiveness determination device the processes, the processes comprising:
acquiring a photographed image of a driver driving a moving object;
detecting a gaze direction of the driver according to the photographed image, and determining whether or not the driver is in an inattentiveness state on a basis of a predetermined gaze range based on a moving direction of the moving object and the gaze direction; and
determining, if the gaze direction cannot be detected, whether or not the driver is in the inattentiveness state on a basis of a face direction of the driver and a predetermined range of the face direction based on the moving direction of the moving object.

### Claim 9 (depends on 7)

The inattentiveness determination method according to claim 7, wherein
in the detecting and determining, a time when the gaze direction is not in the predetermined gaze range is measured and the inattentiveness state is determined if a ratio per unit time of the measured time is equal to or greater than a first predetermined ratio.

### Claim 10 (depends on 7)

The inattentiveness determination method according to claim 7, wherein
in the determining if the gaze direction cannot be detected, a time when the face direction is not in the predetermined range of the face direction is measured and the inattentiveness state is determined if a ratio per unit time of the measured time is equal to or greater than a second predetermined ratio.

### Claim 11 (depends on 7)

The inattentiveness determination method according to claim 7, further comprising
determining, in a case where the gaze direction and the face direction cannot be detected, whether or not the driver's face can be detected on a basis of the photographed image and determine the inattentiveness state if the driver's face cannot be detected.

### Claim 12 (depends on 11)

The inattentiveness determination method according to claim 11, wherein
in the determining in the case where the gaze direction and the face direction cannot be detected, a time when the driver's face cannot be detected is detected, and the inattentiveness state is determined when a ratio per unit time of the detected time is equal to or greater than a third predetermined ratio.

### Claim 13 (depends on 8)

The non-transitory computer-readable storage medium according to claim 8, wherein
in the detecting and determining, a time when the gaze direction is not in the predetermined gaze range is measured and the inattentiveness state is determined if a ratio per unit time of the measured time is equal to or greater than a first predetermined ratio.

### Claim 14 (depends on 8)

The non-transitory computer-readable storage medium according to claim 8, wherein
in the determining if the gaze direction cannot be detected, a time when the face direction is not in the predetermined range of the face direction is measured and the inattentiveness state is determined if a ratio per unit time of the measured time is equal to or greater than a second predetermined ratio.

### Claim 15 (depends on 8)

The non-transitory computer-readable storage medium according to claim 8, further comprising
determining, in a case where the gaze direction and the face direction cannot be detected, whether or not the driver's face can be detected on a basis of the photographed image and determine the inattentiveness state if the driver's face cannot be detected.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium according to claim 15, wherein
in the determining in the case where the gaze direction and the face direction cannot be detected, a time when the driver's face cannot be detected is detected, and the inattentiveness state is determined when a ratio per unit time of the detected time is equal to or greater than a third predetermined ratio.
