US 2021/0201464 A1Published application

Sensor degradation detection and remediation

AssigneeZoox, Inc.

InventorsSarah Tariq, James William Vaisey Philbin, Yi Xu

Life of this patent

Estimating the term…

Loading patent…

Abstract

A sensor degradation detection and remediation system includes one or more sensors configured to collect image data from an environment. A combination of techniques may be used to detect degradations within regions of the image data captured by a sensor, including one or more of determining a level of the visual consistency between the associated image regions captured by different sensors, determining a level of opaqueness of the image regions, and/or measuring temporal movement of the image regions captured by a sensor over a period of time. Operations of a vehicle or other system may be controlled based at least in part on the detection of degradations of the image data captured by the sensors, including automated cleaning of a sensor surface, reducing a level of reliance on the image data received from the sensor, and/or changing a direction of travel of the vehicle.

Claims20

Dependent claims

A vehicle comprising:

one or more processors;
a first sensor to capture sensor data of an environment of the vehicle;
a second sensor to capture sensor data of the environment of the vehicle, the second sensor having a field of view that at least partially overlaps with a field of view of the first sensor;
and
non-transitory computer-readable media storing instructions that when executed by the one or more processors, cause the vehicle to perform operations comprising:
receiving first image data captured by the first sensor at a first time;
identifying a first image region within the first image data;
retrieving second image data captured by the second sensor;
identifying a second image region within the second image data associated with the first image region;
determining a level of visual consistency between the first image region and the second image region;
detecting a degradation of the first image data captured by the first sensor, based on a combination of the determined level of visual consistency and at least one of:
an intensity associated with the first image region;
a measurement of temporal movement associated with the first image region, based on third image data captured at a second time different from the first time;
or
an output from a machine learned model trained to detect degradations based on image data;
and
controlling an operation of the vehicle based at least in part on the detection of the degradation.

14 more elements

A method comprising:

receiving first image data captured by a first sensor of a vehicle at a first time;
identifying a first image region within the first image data;
detecting a degradation of the first image data captured by the first sensor, wherein said detecting is based on at least two of:
a determined level of visual consistency between the first image region and a corresponding second image region within second image data captured by a second sensor;
an intensity associated with the first image region;
a measurement of temporal movement associated with the first image region based on third image data captured at a second time different from the first time;
an output from a machine learned model trained to detect degradations based on image data;
and
controlling an operation of the vehicle based at least in part on the detection of the degradation.

6 more elements

A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving first image data captured by a first sensor of a vehicle at a first time;
identifying a first image region within the first image data;
and
detecting a degradation of the first image data captured by the first sensor, and wherein said detecting is based on at least two of:
a determined level of visual consistency between the first image region and an associated second image region within second image data captured by a second sensor;
an intensity associated with the first image region;
a measurement of temporal movement associated with the first image region based on third image data captured at a second time different from the first time;
an output from a machine learned model trained to detect degradations based on image data;
and
controlling an operation of the vehicle based at least in part on the detecting of the degradation.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/728,532Filed 27 Dec 2019
Published: 1 Jul 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification25 codes

B60W 50/0205
Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures: diagnosing or detecting failures;…
G01S 7/497
Of systems according to group G01S17/00: means for monitoring or calibrating
G01S 17/931
For anti-collision purposes: of land vehicles
G05D 1/0246
Using optical position detecting means: using a video camera in combination with image processing means