# US 2019/0378286 A1 · Zero order light removal in active sensing systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378286 A1
- Document ID: US20190378286A1
- Status: Published application
- Kind: A1
- Application number: 16/408,198
- Filed: 2019-05-09
- Published: 2019-12-12
- Earliest claimed: 2018-06-06
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: James Wilson Nash; Kalin Mitkov Atanassov; Shandon Campbell
- Web page: https://patentgrep.com/patent/US20190378286A1

## CPC

- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 5/20: using local operators
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 2207/10052: Images from lightfield camera
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A device for image processing includes an optical receiver configured to receive a reflection of a coded pattern from an object to generate an image, and processing circuitry. The processing circuitry is configured to determine an estimated position of zero order light in the image, determine a spatial region of the coded pattern that corresponds to a position of the zero order light in the coded pattern, map the spatial region to the estimated position of the zero order light in the image to generate a corrected image, and generate a depth map for the coded pattern based on the corrected image.

## Claims

### Claim 1

A method of image processing, the method comprising:
transmitting, with an optical transmitter, towards an object, a coded pattern of light;
receiving, with an optical receiver, a reflection of the coded pattern from the object to generate an image;
determining, with processing circuitry, an estimated position of zero order light in the image;
determining, with the processing circuitry, a spatial region of the coded pattern that corresponds to a position of the zero order light in the coded pattern;
mapping, with the processing circuitry, the spatial region to the estimated position of the zero order light in the image to generate a corrected image; and
generating, with the processing circuitry, a depth map for the coded pattern based on the corrected image.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the depth map comprises:
determining, with the processing circuitry, a disparity between a transmitting location of the spatial region in the coded pattern and a receiving location of the spatial region in the corrected image; and
generating, with the processing circuitry, the depth map based on the disparity.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining the disparity between the transmitting location of the spatial region and the receiving location of the spatial region comprises:
calculating
d
=
mod

(
X
0
2
-
c


1
,
P
)
,
wherein d is the disparity between the transmitting location of the spatial region and the receiving location of the spatial region, X0 is the estimated position of the of zero order light in the image, c1 is the spatial region of the coded pattern that corresponds to the position of the zero order light in the coded pattern, and P is a period for the spatial region.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
generating, with the processing circuitry, a digital representation of a scene based on the depth map; and
outputting, with the processing circuitry, the digital representation of the scene to a display.

### Claim 5 (depends on 1)

The method of claim 1, wherein the zero order light obscures the spatial region at the position of the zero order light in the image.

### Claim 6 (depends on 1)

The method of claim 1, wherein the zero order light indicates undiffracted light output by the optical transmitter when transmitting the coded pattern.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the estimated position of the zero order light comprises:
determining a probable location window for the zero order light in the image based on at least one of a minimum estimated distance between the optical transmitter and an object reflecting the coded pattern, a maximum estimated distance between the optical transmitter and the object reflecting the coded pattern, or a focal length of the optical receiver.

### Claim 8 (depends on 7)

The method of claim 7, wherein determining the probable location window comprises:
calculating an inner horizontal edge of the probable location window using the minimum estimated distance between the optical transmitter and the object reflecting the coded pattern; and
calculating an outer horizontal edge of the probable location window using the maximum estimated distance between the optical transmitter and the object reflecting the coded pattern.

### Claim 9 (depends on 7)

The method of claim 7, wherein determining the estimated position of the zero order light further comprises:
filtering the probable location window in the image using at least one of a plurality of filters, each filter of the plurality of filters corresponding to a respective target size of the zero order light in the coded pattern.

### Claim 10 (depends on 9)

The method of claim 9, wherein filtering the probable location window comprises applying multiscale blob detection.

### Claim 11 (depends on 9)

The method of claim 9, wherein filtering the coded pattern comprises:
selecting a filter of the plurality of filters based on a difference of Gaussian for the plurality of filters; and
filtering the coded pattern with the selected filter.

### Claim 12 (depends on 1)

The method of claim 1, wherein the spatial region includes an all ‘1’ codeword and wherein mapping the spatial region comprises mapping the spatial region using the all ‘1’ codeword.

### Claim 13 (depends on 1)

The method of claim 1, wherein the spatial region includes an arbitrary codeword and wherein mapping the spatial region comprises mapping the spatial region using the arbitrary codeword.

### Claim 14

A device for image processing, the device comprising:
an optical receiver configured to receive a reflection of a structured light from an object to generate an image, the coded pattern; and
processing circuitry configured to:
determine an estimated position of zero order light in the image;
determine a spatial region of the coded pattern that corresponds to a position of the zero order light in the coded pattern;
map the spatial region to the estimated position of the zero order light in the image to generate a corrected image; and
generate a depth map for the coded pattern based on the corrected image.

### Claim 15 (depends on 14)

The device of claim 14, wherein, to generate the depth map, the processing circuitry is configured to:
determine a disparity between a transmitting location of the spatial region in the coded pattern and a receiving location of the spatial region in the corrected image; and
generate the depth map based on the disparity.

### Claim 16 (depends on 15)

The device of claim 15, wherein, to determine the disparity between the transmitting location of the spatial region and the receiving location of the spatial region, the processing circuitry is configured to:
calculate
d
=
mod

(
X
0
2
-
c


1
,
P
)
,
wherein d is the disparity between the transmitting location of the spatial region and the receiving location of the spatial region, X0 is the estimated position of the of zero order light in the image, c1 is the spatial region of the coded pattern that corresponds to the position of the zero order light in the coded pattern, and P is a period for the spatial region.

### Claim 17 (depends on 14)

The device of claim 14, wherein the processing circuitry is further configured to:
generate a digital representation of a scene based on the depth map; and
output the digital representation of the scene to a display.

### Claim 18 (depends on 14)

The device of claim 14, wherein the zero order light obscures the spatial region at the position of the zero order light in the image.

### Claim 19 (depends on 14)

The device of claim 14, wherein the zero order light indicates undiffracted light output by the optical transmitter when transmitting the coded pattern.

### Claim 20 (depends on 14)

The device of claim 14, wherein, to determine the estimated position of the zero order light, the processing circuitry is configured to:
determine a probable location window for the zero order light in the image based on at least one of a minimum estimated distance between the optical transmitter and an object reflecting the coded pattern, a maximum estimated distance between the optical transmitter and the object reflecting the coded pattern, or a focal length of the optical receiver.

### Claim 21 (depends on 20)

The device of claim 20, wherein, to determine the probable location window, the processing circuitry is configured to:
calculate an inner horizontal edge of the probable location window using the minimum estimated distance between the optical transmitter and the object reflecting the coded pattern; and
calculate an outer horizontal edge of the probable location window using the maximum estimated distance between the optical transmitter and the object reflecting the coded pattern.

### Claim 22 (depends on 20)

The device of claim 20, wherein, to determine the estimated position of the zero order light, the processing circuitry is configured to:
filter the probable location window in the image using at least one of a plurality of filters, each filter of the plurality of filters corresponding to a respective target size of the zero order light in the coded pattern.

### Claim 23 (depends on 22)

The device of claim 22, wherein, to filter the probable location window, the processing circuitry is configured to apply multiscale blob detection.

### Claim 24 (depends on 22)

The device of claim 22, wherein, to filter the probable location window, the processing circuitry is configured to:
select a filter of the plurality of filters based on a difference of Gaussian for the plurality of filters; and
filter the coded pattern with the selected filter.

### Claim 25 (depends on 14)

The device of claim 14, wherein the processing circuitry comprises first processing circuitry, the device further comprising second processing circuitry configured to:
receive the generated depth map from the first processing circuit; and
generate graphical data for the one or more images based on the generated depth map.

### Claim 26 (depends on 25)

The device of claim 25, wherein the first processing circuitry and the second processing circuitry are the same processing circuit.

### Claim 27 (depends on 14)

The device of claim 14, wherein the device comprises one of:
a wireless communication device, a laptop, a desktop, a tablet, a camera, and a video gaming console.

### Claim 28 (depends on 14)

The device of claim 14, comprising:
an optical transmitter configured to transmit, towards the object, the coded pattern.

### Claim 29

A computer-readable storage medium including instructions stored thereon that when executed cause one or more processors of a device for image processing to:
cause an optical transmitter to transmit, towards an object, a coded pattern of light;
cause an optical receiver to receive a reflection of the coded pattern from the object to generate an image; and
cause processing circuitry to:
determine an estimated position of zero order light in the image;
determine a spatial region of the coded pattern that corresponds to a position of the zero order light in the coded pattern;
map the spatial region to the estimated position of the zero order light in the image to generate a corrected image; and
generate a depth map for the coded pattern based on the corrected image.

### Claim 30

A device for image processing, the device comprising:
means for transmitting, towards an object, a coded pattern of light;
means for receiving a reflection of the coded pattern from the object to generate an image;
means for determining an estimated position of zero order light in the image;
means for determining a spatial region of the coded pattern that corresponds to a position of the zero order light in the coded pattern;
means for mapping the spatial region to the estimated position of the zero order light in the image to generate a corrected image; and
means for generating a depth map for the coded pattern based on the corrected image.