# US 11,906,628 B2 · Depth mapping using spatial multiplexing of illumination phase

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,906,628 B2
- Document ID: US11906628B2
- Status: Granted
- Kind: B2
- Application number: 16/939,019
- Filed: 2020-07-26
- Published: 2021-02-18
- Granted: 2024-02-20
- Earliest claimed: 2019-08-15
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Brian S. Medower; Bernhard Buettgen
- Web page: https://patentgrep.com/patent/US11906628B2

## CPC

- G01S 7/4914: of detector arrays, e.g. charge-transfer gates
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G01S 7/4815: {using multiple transmitters}
- G01S 7/4816: {of receivers alone}
- G01S 7/4817: {relating to scanning}
- G01S 7/484: Transmitters
- G01S 7/4863: Detector arrays, e.g. charge-transfer gates
- G01S 7/4911: Transmitters
- G01S 17/10: using transmission of interrupted, pulse-modulated waves (determination of distance by phase measurements G01S17/32)
- G01S 17/14: wherein a voltage or current pulse is initiated and terminated in accordance with the pulse transmission and echo reception respectively, e.g. using counters
- G01S 17/36: with phase comparison between the received signal and the contemporaneously transmitted signal
- G01S 17/89: for mapping or imaging
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

Apparatus for optical sensing includes an illumination assembly, which is configured to direct a first array of beams of optical radiation toward different, respective areas in a target scene while modulating the beams with respective carrier waves having a common carrier frequency and different respective phase angles, which vary across the first array in a predefined spatial pattern. A detection assembly includes a second array of sensing elements, which are configured to output respective signals in response to the optical radiation that is incident on the sensing elements during one or more detection intervals, which are synchronized with the carrier frequency, and objective optics, which are configured to form an image of the target scene on the second array. Processing circuitry processes the signals output by the sensing elements in order to generate a depth map of the target scene.

## Claims

### Claim 1

Apparatus for optical sensing, comprising:
an illumination assembly, which is configured to direct multiple sets of beams of optical radiation toward different, respective areas in a target scene while modulating the beams with respective carrier waves having a common carrier frequency, each set comprising a group of mutually adjacent beams, which are modulated simultaneously at different respective phase angles in a predefined spatial pattern and which are incident on a corresponding cluster of points in the target scene;
a detection assembly, which is configured to receive the optical radiation that is reflected from the target scene, and comprises:
an array of sensing elements, which are configured to output respective signals in response to the optical radiation that is incident on the sensing elements during one or more detection intervals, which are synchronized with the carrier frequency; and
objective optics, which are configured to form an image of the target scene on the array such that clusters of the points in the target scene that are illuminated by respective sets of the beams are imaged onto corresponding groups of the sensing elements, whereby the signals output by the different sensing elements in each of the groups represent the different respective phases of the carrier waves used in modulating the adjacent beams; and
processing circuitry, which is configured to combine the signals output by the sensing elements in each of the groups in order to generate a depth map of the target scene.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein the predefined spatial pattern comprises a grid pattern.

### Claim 3 (depends on 1)

The apparatus according to claim 1, wherein the predefined spatial pattern comprises parallel stripes.

### Claim 4 (depends on 1)

The apparatus according to claim 1, wherein the predefined spatial pattern comprises an irregular variation of the respective phase angles across the first array.

### Claim 5 (depends on 1)

The apparatus according to claim 1, wherein the predefined spatial pattern comprises a quasi-continuous pattern.

### Claim 6 (depends on 1)

The apparatus according to claim 1, wherein the objective optics are configured to form the image of the target scene on the array such that the respective areas in the target scene that are irradiated by the beams having the different respective phase angles are imaged onto corresponding sets of one or more of the sensing elements, and
wherein the processing circuitry is configured to process the signals output by the corresponding sets of the sensing elements responsively to the spatial pattern of the varying phase angles of the carrier waves in order to compute depth coordinates of points in the target scene.

### Claim 7 (depends on 6)

The apparatus according to claim 6, wherein the beams comprise sequences of pulses of the optical radiation, and wherein the signals output by the sensing elements are indicative of respective times of flight of the pulses, and wherein the processing circuitry is configured to process the times of flight together with the different respective phase angles of the beams in computing the depth coordinates.

### Claim 8 (depends on 6)

The apparatus according to claim 6, wherein the detection intervals of the sensing elements occur in multiple different, respective detection phases over the array, and wherein the processing circuitry is configured to apply both the different phase angles of the beams of the optical radiation and the different detection phases of the sensing elements in computing the depth coordinates.

### Claim 9 (depends on 1)

The apparatus according to claim 1, wherein all the sensing elements output the respective signals in response to the optical radiation that is incident on the sensing elements during one detection interval.

### Claim 10 (depends on 1)

The apparatus according to claim 1, wherein each cluster of the points includes four points that are illuminated respectively by four beams having four different, respective phase angles and are imaged onto a respective group of at least four different sensing elements.

### Claim 11 (depends on 10)

The apparatus according to claim 10, wherein the illumination assembly comprises:
four matrices of beam sources, wherein each of the four matrices is configured to generate the beams with a different one of the four phase angles; and
projection optics, which are configured to direct the beams toward the target scene while interleaving the beams from the four matrices to form the sets of the beams having the different respective phase angles.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein the projection optics comprise a diffractive optical element.

### Claim 13 (depends on 1)

The apparatus according to claim 1, wherein the common carrier frequency is a first common carrier frequency, and wherein the illumination assembly is further configured to modulate the beams with the carrier waves at a second common carrier frequency, different from the first common carrier frequency, and the processing circuitry is configured to receive and process the signals output by the sensing elements in at least first and second detection intervals, which are respectively synchronized with the first and second common carrier frequencies.

### Claim 14

A method for optical sensing, comprising:
directing multiple sets of beams of optical radiation toward different, respective areas in a target scene while modulating the beams with respective carrier waves having a common carrier frequency, each set comprising a group of mutually adjacent beams, which are modulated simultaneously at different respective phase angles in a predefined spatial pattern and which are incident on a corresponding cluster of points in the target scene;
forming an image of the target scene on an array of sensing elements, which output respective signals in response to the optical radiation that is incident on the sensing elements during one or more detection intervals, which are synchronized with the carrier frequency, such that clusters of the points in the target scene that are illuminated by respective sets of the beams are imaged onto corresponding groups of the sensing elements, whereby the signals output by the different sensing elements in each of the groups represent the different respective phases of the carrier waves used in modulating the adjacent beams; and
processing the signals output by the sensing elements by combining the signals output by the sensing elements in each of the groups in order to generate a depth map of the target scene.

### Claim 15 (depends on 14)

The method according to claim 14, wherein the predefined spatial pattern is selected from a group of patterns consisting of a grid pattern, parallel stripes, an irregular variation of the respective phase angles across the first array, and a quasi-continuous pattern.

### Claim 16 (depends on 14)

The method according to claim 14, wherein the image of the target scene is formed on the array such that the respective areas in the target scene that are irradiated by the beams having the different respective phase angles are imaged onto corresponding sets of one or more of the sensing elements, and
wherein the signals output by the corresponding sets of the sensing elements are processed responsively to the spatial pattern of the varying phase angles of the carrier waves in order to compute depth coordinates of points in the target scene.

### Claim 17 (depends on 16)

The method according to claim 16, wherein directing the multiple sets of beams comprise directing sequences of pulses of the optical radiation toward the target scene, and wherein the signals output by the sensing elements are indicative of respective times of flight of the pulses, and wherein processing the signals comprises applying the times of flight together with the different respective phase angles of the beams in computing the depth coordinates.

### Claim 18 (depends on 14)

The method according to claim 14, wherein the common carrier frequency is a first common carrier frequency, and wherein the beams are further modulated with the carrier waves at a second common carrier frequency, different from the first common carrier frequency, and wherein processing the signals comprises receiving and processing the signals output by the sensing elements in at least first and second detection intervals, which are respectively synchronized with the first and second common carrier frequencies.