# US 12,596,197 B1 · System for mitigation of sensor veiling glare

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,596,197 B1
- Document ID: US12596197B1
- Status: Granted
- Kind: B1
- Application number: 17/663,719
- Filed: 2022-05-17
- Granted: 2026-04-07
- Earliest claimed: 2022-05-17
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Aravindhan Kumarasamy Krishnan; Shreekant Gayaka; Isabella Talley Lewis
- Web page: https://patentgrep.com/patent/US12596197B1

## CPC

- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G01S 7/4802: {using analysis of echo signal for target characterisation; Target signature; Target cross-section}
- G01S 7/4808: {Evaluating distance, position or velocity data}
- G01S 7/493: Extracting wanted echo signals
- G01S 7/497: Means for monitoring or calibrating
- G01S 17/32: using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
- G01S 17/36: with phase comparison between the received signal and the contemporaneously transmitted signal
- G01S 17/46: Indirect determination of position data
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

An autonomous mobile device (AMD) may use an indirect time of flight (iTOF) sensor to acquire depth information about a physical space. This sensor may experience veiling glare in which multipath interference results in incorrect output of false objects that are not actually present in the physical space. A frame of depth data from the iTOF sensor is processed to determine if veiling glare may be present. A frame with possible veiling glare is then filtered at a pixel level to provide filtered depth data by removing or replacing depth data from pixels that may exhibit veiling glare. These pixels are determined to exhibit veiling glare by comparing their amplitude data with amplitude thresholds associated with an observed distance of that pixel and a specified error tolerance. The filtered data may then be used to operate the AMD.

## Claims

### Claim 1

An autonomous mobile device (AMD), the AMD comprising:
an indirect time of flight (iTOF) sensor comprising a light source and a sensor;
one or more memories storing computer-executable instructions; and
one or more processors to execute the computer-executable instructions to:
acquire first data comprising a plurality of pixels, wherein each pixel of the plurality of pixels comprises data indicative of:
an amplitude of reflected light detected by the sensor, wherein the reflected light is a reflection of light emitted by the light source, and
a distance based on a phase difference between the light emitted by the light source and the reflected light detected by the sensor;
determine, using the first data, a first percentage of the plurality of pixels that:
indicate an amplitude greater than a first threshold amplitude, and
indicate a distance that is less than a first threshold distance;
determine the first percentage is greater than a first threshold percentage;
determine second data based on:
filtering individual pixels of the plurality of pixels using pixel amplitude ratio threshold data; and
operate the AMD based on the second data.

### Claim 2 (depends on 1)

The AMD of claim 1, wherein the one or more processors further execute the computer-executable instructions to:
determine a first set of pixels of the plurality of pixels that:
indicate an amplitude greater than a second threshold amplitude, and
indicate a distance that is less than a second threshold distance;
determine an average distance based on the distances indicated by the first set of pixels; and
retrieve the pixel amplitude ratio threshold data based on the average distance.

### Claim 3 (depends on 1)

The AMD of claim 1, wherein the one or more processors further execute the computer-executable instructions to:
retrieve an amplitude scaling factor that is associated with the iTOF sensor;
determine a first set of pixels of the plurality of pixels that:
indicate an amplitude greater than a second threshold amplitude, and
indicate a distance that is less than a second threshold distance;
determine an average amplitude based on the amplitudes indicated by the first set of pixels;
determine, based on the average amplitude and the amplitude scaling factor, an estimated veiling glare amplitude; and
wherein the filtering is based at least in part on the estimated veiling glare amplitude.

### Claim 4 (depends on 1)

The AMD of claim 1, wherein the one or more processors further execute the computer-executable instructions to:
determine, for a pixel of the plurality of pixels:
a first ratio based on the amplitude indicated by the pixel and an estimated veiling glare amplitude, and
a pixel amplitude ratio threshold value for the pixel retrieved from the pixel amplitude ratio threshold data; and
determine the second data based on a comparison of the first ratio to the pixel amplitude ratio threshold value.

### Claim 5 (depends on 4)

The AMD of claim 4, wherein the pixel amplitude ratio threshold data comprises:
a first pixel amplitude ratio threshold value that is associated with a first distance error value and a first distance; and
a second pixel amplitude ratio threshold value that is associated with a second distance error value and a second distance.

### Claim 6 (depends on 4)

The AMD of claim 4, wherein the one or more processors further execute the computer-executable instructions to:
determine the first ratio is greater than or equal to the pixel amplitude ratio threshold value; and
store the amplitude and distance associated with the pixel as a portion of the second data.

### Claim 7 (depends on 4)

The AMD of claim 4, wherein the one or more processors further execute the computer-executable instructions to:
determine a first set of pixels, wherein each pixel of the first set of pixels has the respective first ratio less than the respective pixel amplitude ratio threshold value;
determine, based on distances associated with the first set of pixels, a curvature of the first set of pixels is zero; and
store the amplitudes and distances of the first set of pixels as a portion of the second data.

### Claim 8 (depends on 7)

The AMD of claim 7, wherein the one or more processors further execute the computer-executable instructions to:
determine the first set of pixels are one or more of:
within a threshold pixel distance of one another, or
adjacent to one another.

### Claim 9 (depends on 4)

The AMD of claim 4, wherein the one or more processors further execute the computer-executable instructions to:
determine a first set of pixels, wherein each pixel of the first set of pixels has the respective first ratio that is less than the respective pixel amplitude ratio threshold value;
determine, based on distances associated with the first set of pixels, a curvature of the first set of pixels is not zero; and
wherein the second data does not include the amplitudes and distances of the first set of pixels.

### Claim 10

A method comprising:
determining first data comprising a plurality of pixels, wherein each pixel comprises data indicative of an amplitude and a distance;
determining a first percentage of the plurality of pixels that:
have an amplitude greater than a first threshold amplitude, and
have a distance that is less than a first threshold distance;
determining the first percentage is greater than a first threshold percentage; and
determining second data based on filtering individual pixels of the plurality of pixels using pixel amplitude ratio threshold data.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
determining, for a pixel of the plurality of pixels:
a first ratio based on the amplitude indicated by the pixel and an estimated veiling glare amplitude, and
a pixel amplitude ratio threshold value for the pixel retrieved from the pixel amplitude ratio threshold data; and
determining the second data based on a comparison of the first ratio to the pixel amplitude ratio threshold value.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
determining the first ratio is greater than or equal to the pixel amplitude ratio threshold value; and
storing the amplitude and distance of the pixel as a portion of the second data.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
determining a first set of pixels, wherein each pixel of the first set of pixels has the respective first ratio less than the respective pixel amplitude ratio threshold value;
determining, based on distances associated with the first set of pixels, a curvature of the first set of pixels is zero; and
storing the amplitudes and distances of the first set of pixels as a portion of the second data.

### Claim 14 (depends on 13)

The method of claim 13, wherein each pixel of the first set of pixels are one or more of:
within a threshold pixel distance of one another, or
adjacent to one another.

### Claim 15

A device comprising:
an indirect time of flight (iTOF) sensor;
one or more memories storing computer-executable instructions; and
one or more processors to execute the computer-executable instructions to:
acquire first data comprising a plurality of pixels, wherein each pixel of the plurality of pixels comprises data indicative of an amplitude and a distance;
determine, based on the amplitudes and the distances of the plurality of pixels, a first value;
based on the first value, determine second data based on filtering individual pixels of the plurality of pixels using pixel amplitude ratio threshold data; and
operate the device based on the second data.

### Claim 16 (depends on 15)

The device of claim 15, wherein the one or more processors further execute the computer-executable instructions to:
determine the first value that is indicative of a first percentage of the plurality of pixels that:
have an amplitude greater than a first threshold amplitude, and
have a distance that is less than a first threshold distance.

### Claim 17 (depends on 15)

The device of claim 15, wherein the one or more processors further execute the computer-executable instructions to:
determine, for a pixel of the plurality of pixels:
a first ratio based on the amplitude indicated by the pixel and an estimated veiling glare amplitude, and
a pixel amplitude ratio threshold value for the pixel retrieved from the pixel amplitude ratio threshold data; and
determine the second data based on a comparison of the first ratio to the pixel amplitude ratio threshold value.

### Claim 18 (depends on 17)

The device of claim 17, wherein the one or more processors further execute the computer-executable instructions to:
determine the first ratio is greater than or equal to the pixel amplitude ratio threshold value; and
store the amplitude and distance of the pixel as a portion of the second data.

### Claim 19 (depends on 17)

The device of claim 17, wherein the one or more processors further execute the computer-executable instructions to:
determine a first set of pixels, wherein each pixel of the first set of pixels has the respective first ratio less than the respective pixel amplitude ratio threshold value;
determine, based on distances associated with the first set of pixels, a curvature of the first set of pixels is zero; and
store the amplitudes and distances of the first set of pixels as a portion of the second data.

### Claim 20 (depends on 17)

The device of claim 17, wherein the one or more processors further execute the computer-executable instructions to:
determine a first set of pixels, wherein each pixel of the first set of pixels has the respective first ratio less than the respective pixel amplitude ratio threshold value;
determine, based on distances associated with the first set of pixels, a curvature of the first set of pixels is not zero; and
wherein the second data does not include the amplitudes and distances of the first set of pixels.