# US 2026/0094288 A1 · Sparse depth imaging with interpolated depth values

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0094288 A1
- Document ID: US20260094288A1
- Status: Published application
- Kind: A1
- Application number: 18/905,039
- Filed: 2024-10-02
- Published: 2026-04-02
- Earliest claimed: 2024-10-02
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Ling ZHU; John Peter GODBAZ; Vishali MOGALLAPU
- Web page: https://patentgrep.com/patent/US20260094288A1

## CPC

- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G01B 11/22: for measuring depth
- G01B 11/2513: {with several lines being projected in more than one direction, e.g. grids, patterns}
- G01S 7/4814: {of transmitters alone}
- G01S 7/4816: {of receivers alone}
- G01S 17/89: for mapping or imaging
- G06T 3/04: Context-preserving transformations, e.g. by using an importance map (panospheric to cylindrical image transformations G06T3/12)
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10048: Infrared image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A method for operating a sparse depth imaging system is presented. The method comprises receiving a depth map of an environment. The depth map comprises a plurality of pixels having locations in an optical sensor coordinate system. A pattern of illuminator dots in an optical source coordinate system is received. Each illuminator dot has a fixed location in a defined plane in the optical source coordinate system. The depth map is projected into a 3D point cloud in the optical sensor coordinate system. Each point in the 3D point cloud is assigned a 2D location in the defined plane. A depth value for each illuminator dot is interpolated based on transformed depth of points in the 3D point cloud. Each illuminator dot is assigned a 3D location in the optical sensor coordinate system. A depth for each illuminator dot is output in the optical sensor coordinate system.

## Claims

### Claim 1

A method for operating a sparse depth imaging system, comprising:
receiving a depth map of an environment, the depth map comprising a plurality of pixels having locations in an optical sensor coordinate system;
receiving a pattern of illuminator dots in an optical source coordinate system, each illuminator dot having a fixed location in a defined plane in the optical source coordinate system;
projecting the depth map of the environment into a 3D point cloud in the optical sensor coordinate system;
assigning each point in the 3D point cloud a 2D location in the illuminator normal plane;
interpolating a depth value for each illuminator dot in the pattern of illuminator dots based on transformed depth of points in the 3D point cloud;
assigning each illuminator dot in the pattern of illuminator dots a 3D location in the optical sensor coordinate system; and
outputting a depth for each illuminator dot in the pattern of illuminator dots in the optical sensor coordinate system.

### Claim 2 (depends on 1)

The method of claim 1, wherein the defined plane is an illuminator normal plane.

### Claim 3 (depends on 2)

The method of claim 2, wherein assigning each point in the 3D point cloud the 2D location in the illuminator normal plane comprises:
transforming the 3D point cloud from the optical sensor coordinate system into the optical source coordinate system in three dimensions.

### Claim 4 (depends on 3)

The method of claim 3, wherein assigning each point in the 3D point cloud a 2D location in the illuminator normal plane further comprises:
projecting the transformed 3D point cloud into the illuminator normal plane.

### Claim 5 (depends on 4)

The method of claim 4, wherein projecting the transformed 3D point cloud into the illuminator normal plane comprises dividing X and Y coordinates for each point by a respective Z coordinate.

### Claim 6 (depends on 1)

The method of claim 1, wherein interpolating the depth value for each illuminator dot in the pattern of illuminator dots based on transformed depth of points in the 3D point cloud comprises:
assigning a depth value for an illuminator dot based on a depth value for a nearest assigned point from the 3D point cloud.

### Claim 7 (depends on 1)

The method of claim 1, wherein assigning each illuminator dot in the pattern of illuminator dots the 3D location in the optical sensor coordinate system comprises:
projecting locations for each illuminator dot into the 3D point cloud.

### Claim 8 (depends on 7)

The method of claim 7, wherein assigning each illuminator dot in the pattern of illuminator dots a 3D location in the optical sensor coordinate system further comprises:
converting projected locations for each illuminator dot into the optical sensor coordinate system.

### Claim 9

A depth imaging system, comprising:
an optical source configured to output modulated structured light comprising a pattern of illuminator dots;
an optical sensor comprising a 2D pixel grid;
a logic subsystem; and
a storage subsystem holding instructions executable by the logic subsystem to:
illuminate an environment using the optical source;
receive reflected illumination at the optical sensor;
generate a depth map of the environment, the depth map comprising a plurality of pixels having locations in an optical sensor coordinate system;
receive the pattern of illuminator dots in an optical source coordinate system, each illuminator dot having a fixed location in an illuminator normal plane;
project the depth map of the environment into a 3D point cloud in the optical sensor coordinate system;
assign each point in the 3D point cloud a 2D location in the illuminator normal plane;
interpolate a depth value for each illuminator dot in the pattern of illuminator dots based on transformed depth of points in the 3D point cloud; and
assign each illuminator dot in the pattern of illuminator dots a 3D location in the optical sensor coordinate system.

### Claim 10 (depends on 9)

The depth imaging system of claim 9, wherein the storage subsystem further holds instructions executable by the logic subsystem to:
output a depth for each illuminator dot in the pattern of illuminator dots in the optical sensor coordinate system.

### Claim 11 (depends on 10)

The depth imaging system of claim 10, wherein assigning each point in the 3D point cloud the 2D location in the illuminator normal plane comprises:
transforming the 3D point cloud from the optical sensor coordinate system into the optical source coordinate system in three dimensions.

### Claim 12 (depends on 11)

The depth imaging system of claim 11, wherein assigning each point in the 3D point cloud the 2D location in the illuminator normal plane further comprises:
projecting the transformed 3D point cloud into the illuminator normal plane.

### Claim 13 (depends on 12)

The depth imaging system of claim 12, wherein projecting the transformed 3D point cloud into the illuminator normal plane comprises dividing X and Y coordinates for each point by a respective Z coordinate.

### Claim 14 (depends on 9)

The depth imaging system of claim 9, wherein interpolating the depth value for each illuminator dot in the pattern of illuminator dots based on transformed depth of points in the 3D point cloud comprises:
assigning a depth value for an illuminator dot based on a depth value for a nearest assigned point from the 3D point cloud.

### Claim 15 (depends on 9)

The depth imaging system of claim 9, wherein assigning each illuminator dot in the pattern of illuminator dots the 3D location in the optical sensor coordinate system comprises:
projecting locations for each illuminator dot into the 3D point cloud.

### Claim 16 (depends on 9)

The depth imaging system of claim 9, wherein assigning each illuminator dot in the pattern of illuminator dots a 3D location in the optical sensor coordinate system further comprises:
converting projected locations for each illuminator dot into the optical sensor coordinate system.

### Claim 17 (depends on 9)

The depth imaging system of claim 9, wherein the depth imaging system is a head-mounted display system.

### Claim 18 (depends on 9)

The depth imaging system of claim 9, wherein the optical source is an infrared (IR) or near-infrared (NIR) illumination source configured to output modulated structured IR or NIR light comprising the pattern of illuminator dots.

### Claim 19

A storage machine holding instructions executable by a logic machine to:
illuminate an environment using an optical source configured to output modulated structured light comprising a pattern of illuminator dots;
receive reflected illumination at an optical sensor comprising a 2D pixel grid;
generate a depth map of the environment, the depth map comprising a plurality of pixels having locations in an optical sensor coordinate system;
receive the pattern of illuminator dots in an optical source coordinate system, each illuminator dot having a fixed location in an illuminator normal plane;
project the depth map of the environment into a 3D point cloud in the optical sensor coordinate system;
assign each point in the 3D point cloud a 2D location in the illuminator normal plane;
interpolate a depth value for each illuminator dot in the pattern of illuminator dots based on transformed depth of points in the 3D point cloud; and
assign each illuminator dot in the pattern of illuminator dots a 3D location in the optical sensor coordinate system.

### Claim 20 (depends on 19)

The storage machine of claim 19, further holding instructions executable by the logic machine to:
output a depth for each illuminator dot in the pattern of illuminator dots in the optical sensor coordinate system.
