# US 2021/0174525 A1 · Time-of-flight depth measurement using modulation frequency adjustment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174525 A1
- Document ID: US20210174525A1
- Status: Published application
- Kind: A1
- Application number: 17/109,439
- Filed: 2020-12-02
- Published: 2021-06-10
- Earliest claimed: 2019-05-02
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: GAL BITAN; ROY YAM; GERSHI KOLTUN
- Web page: https://patentgrep.com/patent/US20210174525A1

## CPC

- G01S 17/32: using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G01S 7/497: Means for monitoring or calibrating
- G01S 17/10: using transmission of interrupted, pulse-modulated waves (determination of distance by phase measurements G01S17/32)
- G01S 17/89: for mapping or imaging
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G06V 40/161: {Detection; Localisation; Normalisation}
- H04N 5/2226: {Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10)}
- H04N 23/56: provided with illuminating means
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10048: Infrared image
- G06T 2207/10152: Varying illumination
- G06T 2207/20104: Interactive definition of region of interest [ROI]
- G06T 2207/30201: Face
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In a method for time-of-flight (ToF) based measurement, a scene is illuminated using a ToF light source modulated at a first modulation frequency FMOD(1). While the light is modulated using FMOD(1), depths are measured to respective surface points within the scene, where the surface points are represented by a plurality of respective pixels. At least one statistical distribution parameter is computed for the depths. A second modulation frequency FMOD(2) higher than FMOD(1) is determined based on the at least one statistical distribution parameter. The depths are then re-measured using FMOD(2) to achieve a higher depth accuracy.

## Claims

### Claim 1

A time-of-flight (ToF) camera comprising:
an interface configured to receive a first ToF light source modulated at a first modulation frequency and a second ToF light source modulated at a second modulation frequency; and
an image sensor coupled to an image signal processor configured to measure first depths of a region of interest (ROI) based on the first ToF light source and second depths of the ROI based on the second ToF light source,
wherein the image signal processor is configured to determine the second modulation frequency based on a measuring result of the first depths.

### Claim 2 (depends on 1)

The ToF camera of claim 1, wherein the ToF camera is configured to illuminate a scene using the first ToF light source and to identify the ROI within the scene.

### Claim 3 (depends on 2)

The ToF camera of claim 2, wherein the RoI is represented by an entire frame of a captured image.

### Claim 4 (depends on 2)

The ToF camera of claim 2, wherein the ToF camera is configured as a 2-tap ToF camera.

### Claim 5 (depends on 2)

The ToF camera of claim 2, wherein the ToF camera is configured as a 4-tap ToF camera.

### Claim 6 (depends on 4)

The ToF camera of claim 4, wherein the image signal processor is configured to compute at least one statistical distribution parameter for the first depths and to determine the second modulation frequency based on the at least one statistical distribution parameter for the first depths.

### Claim 7 (depends on 6)

The ToF camera of claim 6, wherein the at least one statistical distribution parameter for the first depths is the standard deviation of the first depths.

### Claim 8 (depends on 6)

The ToF camera of claim 6, wherein the second modulation frequency is higher than the first modulation frequency.

### Claim 9 (depends on 8)

The ToF camera of claim 8, wherein the first modulation frequency is 20 MHz.

### Claim 10 (depends on 1)

The ToF camera of claim 1, wherein the interface is further configured to receive a third ToF light source modulated at a third modulation frequency, and
wherein the image signal processor is further configured to measure third depths of the ROI based on the third ToF light source after measuring the first depths and the second depths.

### Claim 11 (depends on 10)

The ToF camera of claim 10, wherein the third modulation frequency is higher than the first modulation frequency.

### Claim 12 (depends on 10)

The ToF camera of claim 10, wherein the third modulation frequency is higher than the second modulation frequency, and
wherein the image signal processor is configured to compute at least one statistical distribution parameter for the second depths and to determine the third modulation frequency based on the at least one statistical distribution parameter for the second depths.

### Claim 13

A time-of-flight (ToF) camera comprising:
an interface configured to receive a first ToF light source modulated at a first modulation frequency and a second ToF light source modulated at a second modulation frequency;
an illuminator configured to illuminate a scene using the first ToF light source; and
an image signal processor configured to:
identify a region of interest (ROI) within the scene;
measure first depths of a region of interest (ROI) based on the first ToF light source and second depths of the ROI based on the second ToF light source; and
determine the second modulation frequency based on at least one statistical distribution parameter for the first depths,
wherein the second modulation frequency is different from the first modulation frequency.

### Claim 14 (depends on 13)

The ToF camera of claim 13, wherein the ToF camera is configured as a 2-tap ToF camera.

### Claim 15 (depends on 13)

The ToF camera of claim 13, wherein the first modulation frequency is 20 MHz.

### Claim 16 (depends on 13)

The ToF camera of claim 13, wherein the second modulation frequency is higher than the first modulation frequency.

### Claim 17

A time-of-flight (ToF) camera comprising:
an illuminator operable to illuminate a scene with modulated light;
an image sensor comprising pixels to capture the modulated light reflected from surface points in the scene and output voltages representing the same; and
an image signal processor coupled to the illuminator and image sensor and configured to:
measure depths from the image sensor to surface points within the scene with ToF operations using a first modulation frequency FMOD(1) at which the light is modulated;
compute at least one statistical distribution parameter for the depths;
determine a second modulation frequency FMOD(2) higher than FMOD(1) based on the statistical distribution parameter;
re-measure the depths with the light modulated at FMOD(2);
re-compute the statistical distribution parameter for the depths that were re-measured; and
determine whether to provide the re-measured depths using FMOD(2) as final measured depths.

### Claim 18 (depends on 17)

The ToF camera of claim 17, wherein the image signal processor determines the final measured depths based on whether the re-computed statistical parameter is above or below a threshold corresponding to a target depth accuracy.

### Claim 19 (depends on 18)

The ToF camera of claim 18, wherein if the re-computed statistical parameter is below the threshold, the image signal processor provides the re-measured depths using FMOD(2) as the final measured depths.

### Claim 20 (depends on 18)

The ToF camera of claim 18, wherein if the re-computed statistical parameter is above the threshold, the image signal processor determines a third modulation frequency FMOD(3) higher than FMOD(2) based on the re-computed statistical distribution parameter, and re-measures the depths using FMOD(3) to obtain the final measured depths.
