# US 2026/0241929 A1 · Devices and methods for assisting operation of vehicles based on situational assessment fusing exponential risks (safer) or predictive risk fusion

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0241929 A1
- Document ID: US20260241929A1
- Status: Published application
- Kind: A1
- Application number: 19/649,067
- Filed: 2026-04-15
- Published: 2026-08-20
- Earliest claimed: 2021-12-01
- Assignees: Nauto, Inc.
- Applicants: Nauto, Inc.
- Inventors: Stefan Peter Heck; Benjamin Oren Alpert; Tahmida Binte Mahmud; Mohammad Saad Billah; Ilan Hornstein
- Web page: https://patentgrep.com/patent/US20260241929A1

## CPC

- B60W 30/095: Predicting travel path or likelihood of collision
- G05B 15/02: electric
- B60Q 9/008: {for anti-collision purposes}
- B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering
- B60W 30/0956: {the prediction being responsive to traffic or environmental parameters}
- B60W 40/08: related to drivers or passengers
- B60W 50/0097: {Predicting future conditions}
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- B60W 50/16: Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
- G05B 13/027: {using neural networks only}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G07C 5/04: using counting means or digital clocks
- B60W 2050/143: {Alarm means}
- B60W 2050/146: {Display means}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W 2520/10: Longitudinal speed
- B60W 2540/00: Input parameters relating to occupants
- B60W 2540/223: Posture, e.g. hand, foot, or seat position, turned or inclined
- B60W 2540/225: Direction of gaze
- B60W 2540/229: Attention level, e.g. attentive to driving, reading or sleeping
- B60W 2554/802: Longitudinal distance
- B60W 2556/10: Historical data
- G07C 5/0866: {the electronic data carrier being a digital video recorder in combination with video camera}
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

An apparatus includes: a first sensor configured to provide a first input associated with an environment outside a vehicle; a second sensor configured to provide a second input associated with an operation of the vehicle; and a processing unit having a first-stage processing system and a second-stage processing system; the first-stage processing system configured to receive the first input and the second input, process the first input to obtain a first time series of information, and process the second input to obtain a second time series of information; wherein the second-stage processing system comprises a neural network model configured to receive the first time series of information and a second time series of information in parallel; and wherein the neural network is configured to process the first time series and the second time series to determine a probability of a predicted event associated with an operation of the vehicle.

## Claims

### Claim 1

An apparatus comprising:
a first sensor configured to provide a first time series associated with an environment outside a vehicle;
a second sensor configured to provide a second time series associated with a driver of the vehicle; and
a processing unit configured to provide an output based on the first time series and the second time series;
wherein the first time series indicates a first risk factor, the second time series indicates a second risk factor, and the output provided by the processing unit indicates a combined risk that accounts for the first risk factor and the second risk factor; and
wherein the processing unit is configured to monitor values of the output over time to identify an escalating risky situation.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to identify the escalating risky situation by determining whether the output is increasing non-linearly.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to determine whether the output is increasing exponentially.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to receive the first time series and the second time series in parallel, and to process the first time series and the second time series in parallel.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to combine the first time series and the second time series to form a composite time series.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the composite time series is a single-channel input image.

### Claim 7 (depends on 5)

The apparatus of claim 5, wherein the composite time series is a two-dimensional matrix of data.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the first risk factor indicated by the first time series is below a first risk threshold, the second risk factor indicated by the second time series is below a second risk threshold, and the combined risk is above a combined risk threshold.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the first time series and/or the second time series is encoded.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the processing unit comprises a neural network model configured to process the first time series and/or the second time series.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the first time series is first raw data from the first sensor, and/or wherein the second time series is based on second raw data from the second sensor.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the first time series is based on first raw data from the first sensor, and/or wherein the second time series is based on second raw data from the second sensor.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to predict an imminent collision by monitoring the values of the output, and wherein the processing unit is configured to determine whether a time to impact associated with the predicted imminent collision is less than a time threshold, determine whether the vehicle is being operated based on the vehicle information, and generate a control signal if the time to impact is less than the time threshold, and if the vehicle information indicates that the vehicle is not being operated.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the time threshold is three seconds.

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein the vehicle information indicates whether a brake of the vehicle is being applied.

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein the vehicle information indicates whether a traveling direction of the vehicle is being changed.

### Claim 17 (depends on 1)

The apparatus of claim 1, wherein the second sensor is a camera configured to capture an image of the driver.

### Claim 18 (depends on 1)

The apparatus of claim 1, wherein the first sensor comprises a camera, a Lidar, a radar, or any combination of the foregoing, configured to sense the environment outside the vehicle.

### Claim 19 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to generate a control signal for operating a device, wherein the processing unit is configured to generate the control signal based on the output.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein the device comprises:
a speaker for generating an alarm;
a display or a light-emitting device for providing a visual signal;
a haptic feedback device;
a collision avoidance system; or
a vehicle control for the vehicle.

### Claim 21 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to receive a third time series comprising vehicle information from a vehicle communication system, and wherein the processing unit is configured to provide the output based on the first time series, the second time series, and the third time series.

### Claim 22 (depends on 21)

The apparatus of claim 21, wherein the vehicle information from the vehicle communication system comprises signals regarding braking, pedal position, steering angle, or speed.

### Claim 23 (depends on 1)

The apparatus of claim 1, wherein the processing unit is configured to compare a response of the driver and a reference driver response, the reference driver response representing a good driver response.
