# US 2022/0019823 A1 · Anomalous event detection and/or validation using inherent human behavior

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0019823 A1
- Document ID: US20220019823A1
- Status: Published application
- Kind: A1
- Application number: 16/931,846
- Filed: 2020-07-17
- Published: 2022-01-20
- Earliest claimed: 2020-07-17
- Assignees: Toyota Motor Engineering & Manufacturing North America, Inc.
- Applicants: Toyota Motor Engineering & Manufacturing North America, Inc.
- Inventors: Seyhan Ucar; Baik Hoh; Shalini Keshavamurthy; Kentaro Oguchi
- Web page: https://patentgrep.com/patent/US20220019823A1

## CPC

- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- H04W 4/029: Location-based management or tracking services
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/41: {Interactive pattern learning with a human teacher}
- G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V 10/778: Active pattern-learning, e.g. online learning of image or video features
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- H04W 4/40: for vehicles, e.g. vehicle-to-pedestrians [V2P]
- H04W 4/46: for vehicle-to-vehicle communication [V2V]
- H04W 4/022: {with dynamic range variability}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Anomalous events in a driving environment can be detected and/or validated by leveraging inherent human behavior. A notification of the anomalous event can be received from an initial connected entity in the driving environment. In response, an inquiry can be sent to connected entities spatiotemporally related to the initial connected entity as to whether any person associated with the connected entities is exhibiting an inherent human behavior. When a threshold level of responses to the inquiry are received from the connected entities indicating that an inherent human behavior has been detected, the one or more connected entities can be caused to upload driving scene data for at least a time when the inherent human behavior by the occupant of the connected vehicle was detected.

## Claims

### Claim 1

A method of detecting and/or validating an anomalous event in a driving environment, the method comprising:
receiving, from an initial connected entity in the driving environment, a notification of the anomalous event;
sending an inquiry to one or more connected entities spatiotemporally related to the initial connected entity as to whether any person associated with the one or more connected entities is exhibiting an inherent human behavior; and
when a threshold level of responses to the inquiry are received from one or more of the one or more connected entities indicating that an inherent human behavior has been detected, causing the one or more connected entities to upload driving scene data for at least a time when the inherent human behavior was detected.

### Claim 2 (depends on 1)

The method of claim 1, wherein the initial connected entity is a connected vehicle, and wherein receiving a notification of an anomalous event includes receiving a notification that an inherent human behavior by an occupant of the connected vehicle has been detected.

### Claim 3 (depends on 1)

The method of claim 1, wherein the initial connected entity is a communication device, and wherein receiving a notification of an anomalous event includes receiving a notification that a user has provided an input on the communication device indicating an anomalous event.

### Claim 4 (depends on 1)

The method of claim 1, wherein the inherent human behavior is rubbernecking or near-rubbernecking.

### Claim 5 (depends on 1)

The method of claim 1, wherein the connected entities includes at least one of: one or more connected vehicles, one or more connected infrastructure devices, and one or more connected personal devices.

### Claim 6 (depends on 1)

The method of claim 1, further including analyzing the driving scene data, using machine learning or deep learning techniques, to detect an anomalous event in the driving environment.

### Claim 7 (depends on 6)

The method of claim 6, wherein the anomalous event is one of a car accident or an aggressive driver.

### Claim 8 (depends on 1)

The method of claim 1, further including sending the driving scene data for review by a human to identify an anomalous event in the driving environment.

### Claim 9 (depends on 8)

The method of claim 8, further including receiving, from the human, at least one of annotated driving scene data and instructions, whereby a machine learning or deep learning model learns to capture the anomalous event based on the at least one of annotated driving scene data and instructions.

### Claim 10 (depends on 1)

The method of claim 1, further including determining whether the anomalous event is static or dynamic.

### Claim 11 (depends on 1)

The method of claim 1, further including sending a notification or instructions to one or more connected entities based on the anomalous event.

### Claim 12 (depends on 1)

The method of claim 1, further including sending an inquiry to one or more connected entities that have passed the anomalous event as to whether any person associated with the one or more connected entities exhibited an inherent human behavior when at or near the anomalous event.

### Claim 13 (depends on 1)

The method of claim 1, wherein the threshold level is based on real-time conditions in the driving environment.

### Claim 14

A system for detecting an anomalous event in a driving environment, the system comprising:
one or more processors, the one or more processors being programmed to initiate executable operations comprising:
receiving, from an initial connected entity in the driving environment, a notification of an anomalous event;
sending an inquiry to one or more connected entities spatiotemporally related to the initial connected entity as to whether any person associated with the one or more connected entities is exhibiting an inherent human behavior; and
when a threshold level of responses to the inquiry are received from one or more of the one or more connected entities indicating that an inherent human behavior has been detected, causing the one or more connected entities to upload driving scene data for at least a time when the inherent human behavior was detected.

### Claim 15 (depends on 14)

The system of claim 14, wherein the initial connected entity is a connected vehicle, and wherein receiving a notification of an anomalous event includes receiving a notification that an inherent human behavior by an occupant of the connected vehicle has been detected.

### Claim 16 (depends on 14)

The system of claim 14, wherein the initial connected entity is a communication device, and wherein receiving a notification of an anomalous event includes receiving a notification that a user has provided an input on the communication device indicating an anomalous event.

### Claim 17 (depends on 14)

The system of claim 14, wherein the inherent human behavior is rubbernecking or near-rubbernecking.

### Claim 18 (depends on 14)

The system of claim 14, wherein in the connected entities includes connected vehicles, connected infrastructure devices, and connected personal devices.

### Claim 19 (depends on 14)

The system of claim 14, further including analyzing the driving scene data, using machine learning or deep learning techniques, to detect an anomalous event in the driving environment.

### Claim 20 (depends on 14)

The system of claim 14, further including sending the driving scene data to a human reviewer to identify an anomalous event in the driving environment.

### Claim 21 (depends on 20)

The system of claim 20, further including receiving, from the human, at least one of annotated driving scene data and instructions, whereby a machine learning or deep learning model learns to capture the anomalous event based on the at least one of annotated driving scene data and instructions.

### Claim 22 (depends on 14)

The system of claim 14, further including determining whether the anomalous event is static or dynamic.

### Claim 23 (depends on 14)

The system of claim 14, further including sending a notification or instructions to one or more connected entities based on the anomalous event.

### Claim 24 (depends on 14)

The system of claim 14, further including sending an inquiry to one or more connected entities that have passed the anomalous event as to whether any person associated with the one or more connected entities exhibited an inherent human behavior when at or near the anomalous event.

### Claim 25 (depends on 14)

The system of claim 14, wherein the threshold level of responses is based on real-time conditions in the driving environment.

### Claim 26

A computer program product for detecting and/or validating an anomalous event in a driving environment, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by a processor to perform a method comprising:
receiving, from an initial connected entity in the driving environment, a notification of an anomalous event;
sending an inquiry to one or more connected entities spatiotemporally related to the initial connected entity as to whether any person associated with the one or more connected entities is exhibiting an inherent human behavior; and
when a threshold level of responses indicate that an inherent human behavior has been detected by a person associated with the connected entity, causing the connected entities to upload driving scene data for at least a time when the inherent human behavior was detected.
