# US 2023/0267628 A1 · Decoding an image for active depth sensing to account for optical distortions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267628 A1
- Document ID: US20230267628A1
- Status: Published application
- Kind: A1
- Application number: 18/005,106
- Filed: 2021-09-20
- Published: 2023-08-24
- Earliest claimed: 2020-09-23
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Ioannis NOUSIAS; Matthieu Jean Olivier DUPRE
- Web page: https://patentgrep.com/patent/US20230267628A1

## CPC

- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G01B 11/2513: {with several lines being projected in more than one direction, e.g. grids, patterns}
- G06T 7/11: Region-based segmentation
- G06T 2207/10048: Infrared image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS

## Abstract

Aspects of the disclosure relate to decoding an image for active depth sensing. An example method includes receiving an image. The image includes one or more reflections of a distribution of light. The method also includes sampling a first region of the image using a first sampling grid to generate a first image sample, sampling a second region of the image using a second sampling grid different from the first sampling grid to generate a second image sample, determining a first depth value based on the first image sample, and determining a second depth value based on the second image sample.

## Claims

### Claim 1

A device for active depth sensing, comprising:
a memory; and
one or more processors configured to:
receive an image, the image including one or more reflections of a distribution of light;
sample a first region of the image using a first sampling grid;
sample the first region of the image using a second sampling grid, the second sampling grid being different from the first sampling grid;
determine a first confidence value associated with the first sampling grid and a second confidence value associated with the second sampling grid; and
based on the first confidence value being greater than the second confidence value, select the first sampling grid for use in determining a first depth value for the first region.

### Claim 2 (depends on 1)

The device of claim 1, wherein the distribution of light is a distribution of light points.

### Claim 3 (depends on 1)

The device of claim 1, wherein the one or more processors configured to:
determine a first image sample based on the sampling of the first region of the image using the first sampling grid; and
determine the first depth value for the first region based on the first image sample.

### Claim 4 (depends on 3)

The device of claim 3, wherein the one or more processors are further configured to:
identify in the first region a first codeword in an array of the distribution of light based on the first image sample; and
determine a first disparity based on a location of the first codeword in the array, wherein determining the first depth value is based on the first disparity.

### Claim 5 (depends on 4)

The device of claim 4, wherein the one or more processors configured to:
sample a second region of the image using a third sampling grid to generate a second image sample; and
determine a second depth value based on the second image sample.

### Claim 6 (depends on 1)

The device of claim 1, wherein an arrangement of sampling points of the second sampling grid differs from an arrangement of sampling points of the first sampling grid.

### Claim 7 (depends on 6)

The device of claim 6, wherein the arrangement of sampling points of the first sampling grid includes a first spacing between sampling points of the first sampling grid, and wherein the arrangement of sampling points of the second sampling grid includes a second spacing between sampling points of the second sampling grid.

### Claim 8 (depends on 7)

The device of claim 7, wherein the first spacing and the second spacing are along a baseline axis and an axis orthogonal to the baseline axis, the baseline axis being associated with a transmitter that transmits the distribution of light and a receiver that captures the image.

### Claim 9 (depends on 1)

The device of claim 1, wherein a total number of sampling points of the second sampling grid differs from a total number of sampling points of the first sampling grid.

### Claim 10 (depends on 1)

The device of claim 1, wherein the first sampling grid is an isotropic sampling grid and the second sampling grid is an anisotropic sampling grid.

### Claim 11 (depends on 1)

The device of claim 1, wherein the one or more processors are further configured to:
determine a first image sample based on the sampling of the first region of the image using the first sampling grid;
determine a second image sample based on the sampling of the first region of the image using the second sampling grid;
compare the first image sample and the second image sample; and
select the first image sample to be used for determining the first depth value based on comparing the first image sample and the second image sample.

### Claim 12 (depends on 11)

The device of claim 11, wherein:
to determine the first confidence value associated with the first sampling grid, the one or more processors are configured to determine the first confidence value for the first image sample;
to determine the second confidence value associated with the second sampling grid, the one or more processors are configured to determine the second confidence value for the second image sample; and
to select the first sampling grid for use in determining the first depth value for the first region, the one or more processors are configured to select the first image sample based on the first confidence value being greater than the second confidence value.

### Claim 13 (depends on 1)

The device of claim 1, further comprising a receiver configured to capture the image.

### Claim 14 (depends on 13)

The device of claim 13, further comprising a transmitter configured to transmit the distribution of light, wherein the transmitter is separated from the receiver by a baseline distance along a baseline axis.

### Claim 15 (depends on 1)

The device of claim 1, further comprising one or more signal processors configured to process the image before decoding the processed image by the one or more processors.

### Claim 16 (depends on 1)

The device of claim 1, wherein the one or more processors are configured to generate a depth map based on the image, wherein the depth map includes a plurality of depth values including the first depth value, and wherein the plurality of depth values indicate one or more depths of one or more objects in a scene captured in the image.

### Claim 17

A method for active depth sensing, comprising:
receiving an image including one or more reflections of a distribution of light;
sampling a first region of the image using a first sampling grid;
sampling the first region of the image using a second sampling grid, the second sampling grid being different from the first sampling grid;
determining a first confidence value associated with the first sampling grid and a second confidence value associated with the second sampling grid; and
based on the first confidence value being greater than the second confidence value, selecting the first sampling grid for use in determining a first depth value for the first region.

### Claim 18 (depends on 17)

The method of claim 17, wherein the distribution of light is a distribution of light points.

### Claim 19 (depends on 17)

The method of claim 17, further comprising:
determining a first image sample based on the sampling of the first region of the image using the first sampling grid; and
determining the first depth value for the first region based on the first image sample.

### Claim 20 (depends on 19)

The method of claim 19, further comprising:
identifying in the first region a first codeword in an array of the distribution of light based on the first image sample; and
determining a first disparity based on a location of the first codeword in the array, wherein determining the first depth value is based on the first disparity.

### Claim 21 (depends on 20)

The method of claim 20, further comprising:
sampling a second region of the image using a third sampling grid to generate a second image sample, the third sampling grid being different from at least one of the first sampling grid and the second sampling grid; and
determining a second depth value based on the second image sample.

### Claim 22 (depends on 17)

The method of claim 17, wherein an arrangement of sampling points of the second sampling grid differs from an arrangement of sampling points of the first sampling grid.

### Claim 23 (depends on 22)

The method of claim 22, wherein the arrangement of sampling points of the first sampling grid includes a first spacing between sampling points of the first sampling grid, and wherein the arrangement of sampling points of the second sampling grid includes a second spacing between sampling points of the second sampling grid.

### Claim 24 (depends on 23)

The method of claim 23, wherein the first spacing and the second spacing are along a baseline axis and an axis orthogonal to the baseline axis, the baseline axis being associated with a transmitter that transmits the distribution of light and a receiver that captures the image.

### Claim 25 (depends on 17)

The method of claim 17, wherein a total number of sampling points of the second sampling grid differs from a total number of sampling points of the first sampling grid.

### Claim 26 (depends on 17)

The method of claim 17, wherein the first sampling grid is an isotropic sampling grid and the second sampling grid is an anisotropic sampling grid.

### Claim 27 (depends on 17)

The method of claim 17, further comprising:
determining a first image sample based on the sampling of the first region of the image using the first sampling grid;
determining a second image sample based on the sampling of the first region of the image using the second sampling grid;
comparing the first image sample and the second image sample; and
selecting the first image sample to be used for determining the first depth value based on comparing the first image sample and the second image sample.

### Claim 28 (depends on 27)

The method of claim 27, wherein:
determining the first confidence value associated with the first sampling grid includes determining the first confidence value for the first image sample;
determining the second confidence value associated with the second sampling grid includes determining the second confidence value for the second image sample; and
selecting the first sampling grid for use in determining the first depth value for the first region includes selecting the first image sample based on the first confidence value being greater than the second confidence value.

### Claim 29 (depends on 17)

The method of claim 17, further comprising:
transmitting the distribution of light.

### Claim 30 (depends on 17)

The method of claim 17, further comprising:
generating a depth map based on the image, wherein the depth map includes a plurality of depth values including the first depth value, and wherein the plurality of depth values indicate one or more depths of one or more objects in a scene captured in the image.
