# US 2025/0166434 A1 · Multi-modal model for traffic accident analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166434 A1
- Document ID: US20250166434A1
- Status: Published application
- Kind: A1
- Application number: 18/954,967
- Filed: 2024-11-21
- Published: 2025-05-22
- Earliest claimed: 2023-11-21
- Assignees: Technology Innovation Institute - Sole Proprietorship LLC
- Applicants: Technology Innovation Institute - Sole Proprietorship LLC
- Inventors: Kebin WU; Wenbin LI; Xiaofei XIAO; Ebtesam ALMAZROUEI
- Web page: https://patentgrep.com/patent/US20250166434A1

## CPC

- G07C 5/0866: {the electronic data carrier being a digital video recorder in combination with video camera}
- G06N 20/00: Machine learning
- G06Q 40/08: Insurance
- G06Q 50/40: Business processes related to the transportation industry (shipping G06Q10/083)
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Disclosed herein are system, method, and computer program product embodiments for a model of traffic accident analysis, which incorporates multi-modal input data to automatically reconstruct accident process video with dynamics details and further provide multi-task analysis with multi-modal outputs.

## Claims

### Claim 1

A method for accident analysis, comprising:
receiving one or more encoded sensor readings, wherein the one or more encoded sensor readings comprise data regarding a traffic accident;
aligning the one or more encoded sensor readings, wherein the aligned one or more encoded sensor readings correspond to the traffic accident; and
generating, by a multi-modal model, an output associated with the traffic accident using the one or more encoded sensor readings.

### Claim 2 (depends on 1)

The method of claim 1, wherein the multi-modal model is a single machine learning model configured to receive the one or more encoded sensor readings and create the output.

### Claim 3 (depends on 1)

The method of claim 1, further comprising defining a privacy policy, wherein sensitive data is removed from the traffic accident data according to the privacy policy prior to receiving the one or more encoded sensor readings.

### Claim 4 (depends on 1)

The method of claim 1, wherein the multi-modal model is configured to receive one or more modalities comprising a video, photograph, audio, text, inertial measurement unit data, GPS data, weather data, vehicle type, vehicle sensor data, date, and time.

### Claim 5 (depends on 3)

The method of claim 3, further comprising:
receiving a score representing an assessment of the output;
calculating an error based on a difference between the output and the score; and
adjusting representations of the one or more modalities using the error.

### Claim 6 (depends on 1)

The method of claim 1, wherein the output is a multi-modal output comprising one or more of a three-dimensional video reconstruction of the traffic accident, an insurance claim, a news report, a traffic report, a traffic management recommendation, and emergency response communications.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
training the multi-modal model using a first set of training data, wherein the first set of training data comprises labeled sensor data;
generating, based on the trained multi-modal model using the first set of training data, a pseudo-label and a confidence score for the pseudo-label for a second set of training data comprising unlabeled sensor data;
determining that the confidence score corresponding to the pseudo-label is greater than a predefined threshold;
applying the pseudo-label to the second set of training data; and
training the multi-modal model using the first set of training data and the pseudo-labeled second set of training data.

### Claim 8 (depends on 1)

The method of claim 1, further comprising training the multi-modal model using training data comprising labeled sensor data, pseudo-labeled sensor data, unlabeled sensor data, and weakly-labeled sensor data.

### Claim 9

A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
receive one or more encoded sensor readings, wherein the one or more encoded sensor readings comprise data regarding a traffic accident;
align the one or more encoded sensor readings, wherein the aligned one or more encoded sensor readings correspond to the traffic accident; and
generate, by a multi-modal model, an output associated with the traffic accident using the one or more encoded sensor readings.

### Claim 10 (depends on 9)

The system of claim 9, wherein the encoded sensor readings comprise one or more floating point numbers, and wherein the at least one processor is further configured to receive the one or more encoded sensor readings from one or more edge systems.

### Claim 11 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to define a privacy policy and remove sensitive data from the traffic accident data according to the privacy policy prior to receiving the one or more encoded sensor readings.

### Claim 12 (depends on 9)

The system of claim 9, wherein the at least one processor is configured to receive one or more modalities comprising a video, photograph, audio, text, inertial measurement unit data, GPS data, weather data, vehicle type, vehicle sensor data, date, and time.

### Claim 13 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to:
receive a score representing an assessment of the output;
calculate an error based on a difference between the output and the score; and
adjust representations of the one or more modalities using the error.

### Claim 14 (depends on 9)

The system of claim 9, wherein the output comprises one or more of a three-dimensional video reconstruction of the traffic accident, an insurance claim, a news report, a traffic report, a traffic management recommendation, and emergency response communications.

### Claim 15 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to:
train the multi-modal model using a first set of training data, wherein the first set of training data comprises labeled sensor data;
generate, based on the trained multi-modal model using the first set of training data, a pseudo-label and a confidence score for the pseudo-label for a second set of training data comprising unlabeled sensor data;
determine that the confidence score corresponding to the pseudo-label is greater than a predefined threshold;
apply the pseudo-label to the second set of training data; and
train the multi-modal model using the first set of training data and the pseudo-labeled second set of training data.

### Claim 16 (depends on 9)

The system of claim 9, wherein the at least one processor is further configured to train the multi-modal model using training data comprising labeled sensor data, pseudo-labeled sensor data, unlabeled sensor data, and weakly-labeled sensor data.

### Claim 17

A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
receiving one or more encoded sensor readings, wherein the one or more encoded sensor readings comprise data regarding a traffic accident;
aligning the one or more encoded sensor readings by time, wherein the aligned one or more encoded sensor readings correspond to the traffic accident; and
generating, by a multi-modal model, an output associated with the traffic accident using the one or more encoded sensor readings.

### Claim 18 (depends on 17)

The non-transitory computer-readable device of claim 17, further comprising defining a privacy policy, wherein sensitive data is removed from the traffic accident data according to the privacy policy prior to receiving the one or more encoded sensor readings.

### Claim 19 (depends on 17)

The non-transitory computer-readable device of claim 17, wherein the computing device is configured to receive one or more modalities comprising a video, photograph, audio, text, inertial measurement unit data, GPS data, weather data, vehicle type, vehicle sensor data, date, and time.

### Claim 20 (depends on 17)

The non-transitory computer-readable device of claim 17, wherein the output comprises one or more of a three-dimensional video reconstruction of the traffic accident, an insurance claim, a news report, a traffic report, a traffic management recommendation, and emergency response communications.
