# US 2026/0085936 A1 · Systems and methods for classifying a vehicle maneuver using a spatiotemporal attention selector

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0085936 A1
- Document ID: US20260085936A1
- Status: Published application
- Kind: A1
- Application number: 19/409,491
- Filed: 2025-12-04
- Published: 2026-03-26
- Earliest claimed: 2023-09-08
- Assignees: Verizon Patent and Licensing Inc.
- Applicants: Verizon Patent and Licensing Inc.
- Inventors: Matteo SIMONCINI; Tommaso BIANCONCINI; Luca BRAVI; Leonardo SARTI; Leonardo TACCARI; Douglas COIMBRA DE ANDRADE; Francesco SAMBO
- Web page: https://patentgrep.com/patent/US20260085936A1

## CPC

- G01C 21/1656: {with passive imaging devices, e.g. cameras}
- B60W 40/09: Driving style or behaviour
- G01C 21/3697: {Output of additional, non-guidance related information, e.g. low fuel level (G01C21/3679 takes precedence)}
- G01S 19/47: the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A device may receive video data and corresponding GPS data and IMU data associated with a vehicle, and may process the video data, with an object detector model, to identify objects and to generate a first feature vector. The device may process the GPS data and the IMU data, with a first CNN model, to generate a second feature vector, and may process the objects and the video data, with a tracking model, to identify positions and classes of the objects and to generate a third feature vector. The device may utilize a second CNN model to generate a matrix of object features based on the first, second, and third feature vectors, and may utilize a spatiotemporal attention selector model or a max pooled model with the matrix of object features to identify a classification of a maneuver of the vehicle. The device may perform actions based on the classification.

## Claims

### Claim 1

A method, comprising:
receiving, by a device, video data, corresponding global positioning system (GPS) data and inertial measurement unit (IMU) data associated with a vehicle;
identifying, by the device, objects in the video data and generating a first feature vector based on the objects;
aligning the GPS data and the IMU data with timestamps of the video data;
generating, by the device, a second feature vector based on the aligned GPS data and the IMU data;
identifying, by the device, positions and classes of the objects and generating a third feature vector based on the positions and the classes;
generating, by the device, a matrix of object features based on the first feature vector, the second feature vector, and the third feature vector;
identifying, by the device, using a machine learning model with the matrix, a classification of a maneuver of the vehicle; and
performing, by the device, one or more actions based on the classification.

### Claim 2 (depends on 1)

The method of claim 1, wherein the aligning comprises:
aligning, using a first CNN model, sampling rates of the GPS data and the IMU data and timestamps of the video data, to extract GPS and IMU features; and
generating the second feature vector based on aligning the sampling rates and the timestamps and based on the GPS and IMU features.

### Claim 3 (depends on 2)

The method of claim 2, wherein the first CNN model includes two-dimensional depth-wise separable convolutions.

### Claim 4 (depends on 1)

The method of claim 1, wherein identifying the objects utilizes an object detector model that includes a faster region-based convolutional neural network model and a residual neural network model, and wherein the object detector model utilizes a pooling layer that computes object features with a single backbone forward pass during a training and an inference phase.

### Claim 5 (depends on 1)

The method of claim 1, wherein the third feature vector is generated using a greedy tracking model that utilizes the positions, the classes, and object detection confidence.

### Claim 6 (depends on 1)

The method of claim 1, wherein identifying the classification further comprises:
classifying the maneuver directly using the machine learning model.

### Claim 7 (depends on 1)

The method of claim 1, wherein identifying the classification further comprises:
processing the matrix of object features with the machine learning model.

### Claim 8 (depends on 7)

The method of claim 7, wherein the maneuver is classified as safe or unsafe and the model is one of a spatiotemporal attention selector model or a max pooled model.

### Claim 9

A device, comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the device to:
receive video data, and corresponding global positioning system (GPS) data and inertial measurement unit (IMU) data associated with a vehicle;
identify objects in the video data and generate a first feature vector based on the objects; align the GPS data and the IMU data with timestamps of the video data;
generate a second feature vector based on the aligned GPS data and the aligned IMU data; identify positions and classes of the objects and generate a third feature vector based on the positions and the classes;
generate a matrix of object features based on the first feature vector, the second feature vector, and the third feature vector;
identify, using a machine learning model with the matrix, a classification of a maneuver of the vehicle; and
perform one or more actions based on the classification.

### Claim 10 (depends on 9)

The device of claim 9, wherein the instructions to align the GPS data and the IMU data cause the device to:
align, using a first CNN model, sampling rates of the GPS data and the IMU data and timestamps of the video data, to extract GPS and IMU features; and
generate the second feature vector based on aligning the sampling rates and the timestamps and based on the GPS and IMU features.

### Claim 11 (depends on 10)

The device of claim 10, wherein the first CNN model includes two-dimensional depth-wise separable convolutions.

### Claim 12 (depends on 9)

The device of claim 9, wherein the instructions to identify the objects utilize an object detector model that includes a faster region-based convolutional neural network model and a residual neural network model, and wherein the object detector model utilizes a pooling layer that computes object features with a single backbone forward pass during a training and an inference phase.

### Claim 13 (depends on 9)

The device of claim 9, wherein the third feature vector is generated using a greedy tracking model that utilizes the positions, the classes, and object detection confidence.

### Claim 14 (depends on 9)

The device of claim 9, wherein the instructions to identify the classification further cause the device to classify the maneuver directly using the machine learning model.

### Claim 15 (depends on 14)

The device of claim 14, wherein the maneuver is classified as safe or unsafe and the machine learning model is one of a spatiotemporal attention selector model or a max pooled model.

### Claim 16

A non-transitory computer-readable medium storing instructions, the instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to:
receive video data, and corresponding global positioning system (GPS) data and inertial measurement unit (IMU) data associated with a vehicle;
identify objects in the video data and generate a first feature vector based on the objects;
align the GPS data and the IMU data with timestamps of the video data;
generate a second feature vector based on the aligned GPS data and the aligned IMU data; identify positions and classes of the objects and generate a third feature vector based on the positions and the classes;
generate a matrix of object features based on the first feature vector, the second feature vector, and the third feature vector;
identify, using a machine learning model with the matrix, a classification of a maneuver of the vehicle; and
perform one or more actions based on the classification.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the one or more instructions to align the GPS data and the IMU data cause the one or more processors to:
align, using a first CNN model, sampling rates of the GPS data and the IMU data and timestamps of the video data, to extract GPS and IMU features; and
generate the second feature vector based on aligning the sampling rates and the timestamps and based on the GPS and IMU features.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the first CNN model includes two-dimensional depth-wise separable convolutions.

### Claim 19 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein identifying the objects utilizes an object detector model that includes a faster region-based convolutional neural network model and a residual neural network model, and wherein the object detector model utilizes a pooling layer that computes object features with a single backbone forward pass during a training and an inference phase.

### Claim 20 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the maneuver is classified as safe or unsafe and the machine learning model is one of a spatiotemporal attention selector model or a max pooled model.
