# US 2021/0334558 A1 · Method and system for providing behavior of vehicle operator using virtuous cycle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334558 A1
- Document ID: US20210334558A1
- Status: Published application
- Kind: A1
- Application number: 17/372,279
- Filed: 2021-07-09
- Published: 2021-10-28
- Earliest claimed: 2016-06-13
- Assignees: Xevo Inc.
- Applicants: Xevo Inc.
- Inventors: Robert Victor Welland; Samuel James McKelvie; Richard Chia-Tsing Tong; Noah Harrison Fradin; Vladimir Sadovsky
- Web page: https://patentgrep.com/patent/US20210334558A1

## CPC

- G06N 20/00: Machine learning
- G06F 18/256: {of results relating to different input data, e.g. multimodal recognition}
- G06V 10/772: Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/811: {the classifiers operating on different input data, e.g. multi-modal recognition}
- G06V 10/95: {structured as a network, e.g. client-server architectures}
- G06V 20/586: {of parking space}
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- B60R 2300/105: using multiple cameras
- B60R 2300/30: characterised by the type of image processing
- B60R 2300/50: characterised by the display information being shared, e.g. external display, data transfer to other traffic participants or centralised traffic controller
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A method or system is capable of detecting operator behavior (“OB”) utilizing a virtuous cycle containing sensors, machine learning center (“MLC”), and cloud based network (“CBN”). In one aspect, the process monitors operator body language captured by interior sensors and captures surrounding information observed by exterior sensors onboard a vehicle as the vehicle is in motion. After selectively recording the captured data in accordance with an OB model generated by MLC, an abnormal OB (“AOB”) is detected in accordance with vehicular status signals received by the OB model. Upon rewinding recorded operator body language and the surrounding information leading up to detection of AOB, labeled data associated with AOB is generated. The labeled data is subsequently uploaded to CBN for facilitating OB model training at MLC via a virtuous cycle.

## Claims

### Claim 1

A system of a vehicle, comprising:
at least one sensor attached to the vehicle and configured to capture data;
a driving behavior module configured to employ a machine learning model to detect dangerous driving behavior based on a triggering event;
a memory that stores computer instructions; and
a processor that is configured to execute the computer instructions to:
monitor driving behavior of a driver of the vehicle based on an analysis of the data captured by the at least one sensor;
detect the triggering event based on the driving behavior; and
provide resistance to at least one component of the vehicle to alert the driver of the dangerous driving behavior detected by the driving behavior module from the triggering event.

### Claim 2 (depends on 1)

The system of claim 1, wherein the processor provides resistance to at least one component of the vehicle by being configured to execute further computer instructions to:
resist erratic steering behavior in response to the driving behavior module detecting dangerous driving.

### Claim 3 (depends on 1)

The system of claim 1, wherein the processor provides resistance to at least one component of the vehicle by being configured to execute further computer instructions to:
provide resistance to a steering wheel of the vehicle in response to the driving behavior module detecting dangerous driving.

### Claim 4 (depends on 1)

The system of claim 1, wherein the processor provides resistance to at least one component of the vehicle by being configured to execute further computer instructions to:
provide resistance to a gas pedal of the vehicle in response to the driving behavior module detecting dangerous driving.

### Claim 5 (depends on 1)

The system of claim 1, wherein the processor detects the triggering event by being configured to execute further computer instructions to:
capture surrounding information observed by at least one exterior sensor of the at least one sensor; and
detect the triggering event based on a combination of the driving behavior and the surrounding information.

### Claim 6 (depends on 1)

The system of claim 1, wherein the processor detects the triggering event by being configured to execute further computer instructions to:
detect rapid steering events based on the data captured from at least one interior sensor.

### Claim 7 (depends on 1)

The system of claim 1, wherein the processor is configured to execute further computer instructions to:
monitor body language of the driver using an interior camera of the at least one sensor to detect operator body movement.

### Claim 8 (depends on 1)

The system of claim 1, wherein the processor is configured to execute further computer instructions to:
employ a steering sensor of the at least one sensor to detect movement of a steering wheel.

### Claim 9

A computing device in a vehicle, comprising:
a memory that stores computer instructions; and
a processor that is configured to execute the computer instructions to:
capture data from at least one interior sensor of the vehicle;
monitor steering behavior of a driver of the vehicle based on an analysis of the data captured by the at least one interior sensor;
detect a triggering event based on the steering behavior;
employ a machine learning model to detect erratic steering behavior based on the triggering event and the monitored steering behavior;
label the erratic steering behavior as a dangerous driving event; and
cause the vehicle to resist the erratic steering behavior in response to the dangerous driving event.

### Claim 10 (depends on 9)

The computing device of claim 9, wherein the processor detects the triggering event by being configured to execute further computer instructions to:
capture surrounding information observed by at least one exterior sensor of the vehicle; and
detect the triggering event based on a combination of the steering behavior and the surrounding information.

### Claim 11 (depends on 9)

The computing device of claim 9, wherein the processor detects the triggering event by being configured to execute further computer instructions to:
detect rapid steering events based on the data captured from the at least one sensor.

### Claim 12 (depends on 9)

The computing device of claim 9, wherein the processor captures data from the at least one interior sensor by being configured to execute further computer instructions to:
monitor body language of the driver using an interior camera to detect operator body movement.

### Claim 13 (depends on 9)

The computing device of claim 9, wherein the processor captures data from the at least one interior sensor by being configured to execute further computer instructions to:
employ a steering sensor to detect movement of a steering wheel.

### Claim 14

A computer implemented method, comprising:
capturing data from at least one interior sensor of a vehicle;
monitoring steering behavior of a driver of the vehicle based on an analysis of the data captured by the at least one interior sensor;
detecting a triggering event based on the steering behavior;
employing a machine learning model to detect erratic steering behavior based on the triggering event and the monitored steering behavior; and
resisting the erratic steering behavior in response to detecting erratic steering behavior.

### Claim 15 (depends on 14)

The method of claim 14, wherein detecting the triggering event comprises:
capturing surrounding information observed by at least one exterior sensor of the vehicle; and
detecting the triggering event based on a combination of the steering behavior and the surrounding information.

### Claim 16 (depends on 15)

The method of claim 15, wherein the detecting the triggering event comprises:
detecting rapid steering events based on the data captured from the at least one sensor.

### Claim 17 (depends on 15)

The method of claim 15, wherein capturing data from the at least one interior sensor comprises:
monitoring body language of the driver using an interior camera to detect operator body movement.

### Claim 18 (depends on 15)

The method of claim 15, wherein capturing data from the at least one interior sensor comprises:
employing a steering sensor to detect movement of a steering wheel.
