US 12,762,865 B2Granted

Occlusion and uncertainty sensitive model training for vehicle applications

AssigneeQUALCOMM Incorporated

InventorsMadhumitha Sakthi, Amin Ansari, Diptiben Navinchandra Patel

Life of this patent

Estimating the term…

Loading patent…

Abstract

This disclosure provides systems, methods, and devices for machine learning techniques for improved training, such as for vehicle surroundings analysis. In one aspect, a method is provided that includes receiving image data and position data from the area surrounding a vehicle, determining initial feature data based on the received data, and determining updated feature data for training a first model. The updated feature data may be determined based on uncertainty measures for portions of the initial feature data, occluded regions within the initial feature data, or combinations thereof. In certain aspects, the first model may be trained using knowledge distillation techniques. Other aspects and features are also claimed and described.

Claims28

Dependent claims

A method comprising:

receiving image data and position data captured from an area surrounding a vehicle;
determining, by a first model based on the image data and the position data, initial feature data for the area surrounding the vehicle;
determining uncertainty measures corresponding to portions of the initial feature data, wherein the uncertainty measures are determined based on error variance in the image data;
determining updated feature data for at least a subset of the initial feature data based on the initial feature data and the uncertainty measures, wherein determining the updated feature data comprises determining updated perspective features for the initial feature data, wherein the updated feature data is determined using knowledge distillation, the knowledge distillation being weighted between the initial feature data and the position data based on the uncertainty measures, wherein the knowledge distillation is weighted based on the uncertainty measures such that:
when the uncertainty measures indicating a confidence meets a first threshold, the knowledge distillation is performed with a higher weight towards the initial feature data;
and
when the uncertainty measures indicating the confidence meets a second threshold, the knowledge distillation is performed with a higher weight towards the position data;
and
training a second model based on the updated feature data.

6 more elements

A method comprising:

receiving image data and position data captured from an area surrounding a vehicle;
determining, by a first model based on the image data and the position data, initial feature data for the area surrounding the vehicle;
determining occluded regions within the image data;
determining uncertainty measures corresponding to portions of the initial feature data, wherein the uncertainty measures are determined based on error variance in the image data;
determining updated feature data for objects corresponding to the occluded regions, wherein determining the updated feature data comprises determining updated top-view features for the initial feature data, wherein determining the updated feature data comprises determining the updated feature data for at least a subset of the initial feature data based on the initial feature data and the uncertainty measures using knowledge distillation, wherein the knowledge distillation is weighted between the initial feature data and the position data based on the uncertainty measures, wherein the knowledge distillation is weighted based on the uncertainty measures such that:
when the uncertainty measures indicating a confidence meets a first threshold, the knowledge distillation is performed with a higher weight towards the initial feature data;
and
when the uncertainty measures indicating the confidence meets a second threshold, the knowledge distillation is performed with a higher weight towards the position data;
and
training a second model based on the updated feature data.

7 more elements

An apparatus, comprising:

an image sensor configured to capture image data for an area surrounding a vehicle;
a position sensor configured to capture position data for the area surrounding the vehicle;
a memory storing processor-readable code;
and
at least one processor coupled to the memory, the image sensor, and the position sensor, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including:
receiving image data and position data captured from the area surrounding the vehicle;
determining, by a first model based on the image data and the position data, initial feature data for the area surrounding the vehicle;
determining uncertainty measures corresponding to portions of the initial feature data, wherein the uncertainty measures are determined based on error variance in the image data;
determining updated feature data for at least a subset of the initial feature data based on the initial feature data and the uncertainty measures, wherein determining the updated feature data comprises determining updated perspective features for the initial feature data, wherein the updated feature data is determined using knowledge distillation, the knowledge distillation being weighted between the initial feature data and the position data based on the uncertainty measures, wherein the knowledge distillation is weighted based on the uncertainty measures such that:
when the uncertainty measures indicating a confidence meets a first threshold, the knowledge distillation is performed with a higher weight towards the initial feature data;
and
when the uncertainty measures indicating the confidence meets a second threshold, the knowledge distillation is performed with a higher weight towards the position data;
and
training a second model based on the updated feature data.

11 more elements

An apparatus, comprising:

an image sensor configured to capture image data for an area surrounding a vehicle;
a position sensor configured to capture position data for the area surrounding the vehicle;
a memory storing processor-readable code;
and
at least one processor coupled to the memory, the image sensor, and the position sensor, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including:
receiving image data and position data captured from the area surrounding the vehicle;
determining, by a first model based on the image data and the position data, initial feature data for the area surrounding the vehicle;
determining occluded regions within the image data;
determining uncertainty measures corresponding to portions of the initial feature data, wherein the uncertainty measures are determined based on error variance in the image data;
determining updated feature data for objects corresponding to the occluded regions, wherein determining the updated feature data comprises determining updated top-view features for the initial feature data, wherein determining the updated feature data comprises determining the updated feature data for at least a subset of the initial feature data based on the initial feature data and the uncertainty measures using knowledge distillation, wherein the knowledge distillation is weighted between the initial feature data and the position data based on the uncertainty measures, wherein the knowledge distillation is weighted based on the uncertainty measures such that:
when the uncertainty measures indicating a confidence meets a first threshold, the knowledge distillation is performed with a higher weight towards the initial feature data;
and
when the uncertainty measures indicating the confidence meets a second threshold, the knowledge distillation is performed with a higher weight towards the position data;
and
training a second model based on the updated feature data.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/745,728Filed 17 Jun 2024
Published: US 2025/0384694 A118 Dec 2025
Granted: 13 Oct 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification4 codes

G06V 20/56
Context or environment of the image: exterior to a vehicle by using sensors mounted on the vehicle
G06V 10/26
Image preprocessing: segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g.…
G06V 10/44
Extraction of image or video features: local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops,…
G06V
Image and video recognition