# US 2024/0198904 A1 · Systems and methods for automated driver assistance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0198904 A1
- Document ID: US20240198904A1
- Status: Published application
- Kind: A1
- Application number: 18/068,742
- Filed: 2022-12-20
- Published: 2024-06-20
- Earliest claimed: 2022-12-20
- Assignees: Huawei Technologies Co., Ltd.
- Inventors: Juntao YE; Yu ZHAO
- Web page: https://patentgrep.com/patent/US20240198904A1

## CPC

- B60Q 9/00: Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 {, e.g. haptic signalling}
- G07C 5/02: Registering or indicating driving, working, idle, or waiting time only (apparatus forming part of taximeters G07B13/00)
- G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08)
- B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

Systems and methods for alerting a driver to a potential driving hazard are disclosed. Hazard parameters representing a driving hazard may be received. The driving hazard may have a corresponding Ground Truth Operation Sequence (GTOS) comprising an ordered sequence of GTOS Designated Meta Operations (DMOs). The corresponding GTOS may be retrieved and instantiated using the hazard parameters. Current driver operation parameters may then be received, and may then be mapped to a GTOS DMO of the ordered sequence of GTOS DMOs. Subsequent driver operation parameters may then be received, and may then be compared to a set of probable subsequent operations associated with the mapped GTOS DMO. It may then be determined that the subsequent driver operation is absent from the set of probable subsequent operations, and an output may be generated for alerting a driver to the driving hazard.

## Claims

### Claim 1

A vehicle control system comprising:
one or more processors;
a memory coupled to the one or more processors, the memory storing executable instructions which, when executed by the one or more processors, cause the system to:
receive, from a driver assistance system, current driver operation parameters representing a current driver operation;
map the current driver operation parameters to a GTOS DMO of an ordered sequence of GTOS DMOs;
receive, from the driver assistance system, subsequent driver operation parameters representing a subsequent driver operation;
compare the subsequent driver operation parameters to a set of probable subsequent operations associated with the mapped GTOS DMO;
determine that the subsequent driver operation is absent from the set of probable subsequent operations; and
generate an output for alerting a driver to the potential driving hazard.

### Claim 2 (depends on 1)

The system of claim 1, before the step of map the current driver operation parameters to a GTOS DMO of an ordered sequence of GTOS DMOs, the system is further caused to: receive, from an exterior environment sensing system, hazard parameters representing a potential driving hazard, the potential driving hazard having a corresponding Ground Truth Operation Sequence (GTOS) comprising an ordered sequence of GTOS designated meta operations (DMOs); retrieve the corresponding GTOS; and instantiate the corresponding GTOS using the hazard parameters.

### Claim 3 (depends on 2)

The system of claim 2, wherein the GTOS is defined by a task data structure, the task data structure comprising the GTOS DMOs and a set of rules, the set of rules defining orders of operation among the GTOS DMOs.

### Claim 4 (depends on 3)

The system of claim 3, wherein each GTOS DMO and each current driver operation DMO includes an operation and one or more operational parameters.

### Claim 5 (depends on 3)

The system of claim 3, wherein to determine the corresponding GTOS, prior to receiving, from the exterior environment sensing system, the hazard parameters representing the potential driving hazard, the potential driving hazard having a corresponding GTOS, the system is further caused to:
receive a plurality of sets of sequential operational parameters representing experienced driver behavior associated with the potential driving hazard;
construct a sequence of DMOs for each set of the plurality of sets of sequential operational parameters;
construct a database of constructed operation sequences based on the constructed sequences of DMOs, wherein each of the constructed operation sequences corresponds to at least one of the constructed sequences of DMOs;
extract a maximal sequential pattern from the database of constructed operation sequences using sequential pattern mining; and
determine the GTOS to be the maximal sequential pattern.

### Claim 6 (depends on 5)

The system of claim 5, wherein the plurality of sets of sequential operational parameters are received from the driver assistance system during operation of the vehicle by experienced drivers.

### Claim 7 (depends on 5)

The system of claim 5, wherein the sequence of DMOs includes multimodality DMOs, and wherein the system is further caused to:
prior to extracting the maximal sequential pattern from the database of constructed operation sequences, linearize each multimodality DMO.

### Claim 8 (depends on 5)

The system of claim 5, wherein the set of probable subsequent operations associated with the mapped GTOS DMO is determined using sequential rule mining on the constructed operation sequences to determine a likely operation following a previous operation.

### Claim 9 (depends on 1)

The system of claim 1, wherein the current driver operation parameters and the subsequent driver operation parameters include one or more gaze parameters, steering parameters, braking parameters and acceleration parameters.

### Claim 10 (depends on 9)

The system of claim 9, wherein the gaze parameters are received from one or more gaze tracking systems, and wherein the steering parameters, the braking parameters and the acceleration parameters are received from one or more Electronic Control Units (ECUs).

### Claim 11

A method for alerting a driver to a potential driving hazard, the method comprising:
receiving, from a driver assistance system, current driver operation parameters representing a current driver operation;
mapping the current driver operation parameters to one of the GTOS DMOs of an ordered sequence of GTOS DMOS;
receiving, from the driver assistance system, subsequent driver operation parameters representing a subsequent driver operation;
comparing the subsequent driver operation parameters to a set of probable subsequent operations associated with the one of the GTOS DMOs;
determining that the subsequent driver operation is absent from the set of probable subsequent operations; and
generating an output for alerting a driver to the potential driving hazard.

### Claim 12 (depends on 11)

The method of claim 11, before mapping the current driver operation parameters to one of the GTOS DMOs of the ordered sequence of GTOS DMOs, further comprising: receiving, from an exterior environment sensing system, hazard parameters representing a potential driving hazard, the potential driving hazard having a corresponding Ground Truth Operation Sequence (GTOS) comprising an ordered sequence of GTOS designated meta operations (DMOs); retrieving the corresponding GTOS; and instantiating the corresponding GTOS using the hazard parameters.

### Claim 13 (depends on 12)

The method of claim 12, wherein the GTOS is defined by a task data structure, the task data structure comprising the GTOS DMOs and a set of rules, the set of rules defining orders of operation among the GTOS DMOs.

### Claim 14 (depends on 12)

The method of claim 12, wherein each GTOS DMO and each current driver operation DMO includes an operation and one or more operational parameters.

### Claim 15 (depends on 12)

The method of claim 12, wherein prior to receiving, from the exterior environment sensing system, the hazard parameters representing the potential driving hazard, the potential driving hazard having a corresponding GTOS, the method further comprises:
receiving a plurality of sets of sequential operational parameters representing experienced driver behavior associated with the potential driving hazard;
constructing a sequence of DMOs for each set of the plurality of sets of sequential operational parameters;
constructing a database of constructed operation sequences based on the constructed sequences of DMOs, wherein each of the constructed operation sequences corresponds to at least one sequence of DMOs;
extracting a maximal sequential pattern from the database of constructed operation sequences using sequential pattern mining; and
determining the GTOS to be the maximal sequential pattern.

### Claim 16 (depends on 15)

The method of claim 15, wherein the plurality of sets of sequential operational parameters are received from the driver assistance system during operation of a vehicle by experienced drivers.

### Claim 17 (depends on 15)

The method of claim 15, wherein the sequence of DMOs includes multimodality DMOs, and wherein the method further comprises:
prior to extracting the maximal sequential pattern from the database of constructed operation sequences, linearizing each multimodality DMO.

### Claim 18 (depends on 15)

The method of claim 15, wherein the set of probable subsequent operations associated with the mapped GTOS DMO is determined using sequential rule mining on the constructed operation sequences to determine a likely operation following a previous operation.

### Claim 19

A non-transitory computer readable medium containing instructions which, which executed by one or more processors, cause the one or more processors to:
receive, from a driver assistance system, current driver operation parameters representing a current driver operation;
map the current driver operation parameters to one of the GTOS DMOs of the ordered sequence of GTOS DMOs;
receive, from the driver assistance system, subsequent driver operation parameters representing a subsequent driver operation;
compare the subsequent driver operation parameters to a set of probable subsequent operations associated with the one of the GTOS DMOs;
determine that the subsequent driver operation is absent from the set of probable subsequent operations; and
generate an output for alerting a driver to the potential driving hazard.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, before the step of map the current driver operation parameters to one of the GTOS DMOs of the ordered sequence of GTOS DMOs; further cause the one or more processors to: receive, from an exterior environment sensing system, hazard parameters representing a potential driving hazard, the potential driving hazard having a corresponding Ground Truth Operation Sequence (GTOS) comprising an ordered sequence of GTOS designated meta operations (DMOs); retrieve the corresponding GTOS; and instantiate the corresponding GTOS using the hazard parameters.
