# US 2023/0410335 A1 · Electronic device for generating depth map and operation method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410335 A1
- Document ID: US20230410335A1
- Status: Published application
- Kind: A1
- Application number: 18/233,611
- Filed: 2023-08-14
- Published: 2023-12-21
- Earliest claimed: 2022-06-21
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Serhii KOLIIEV; Andrii BUGAIOV; Andrii HYRYLA; Andriy BEGUN
- Web page: https://patentgrep.com/patent/US20230410335A1

## CPC

- G06T 7/507: from shading (G06T7/586 takes precedence)
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G01B 11/2513: {with several lines being projected in more than one direction, e.g. grids, patterns}
- G01B 11/2518: {Projection by scanning of the object}
- G06T 7/55: from multiple images
- G06T 7/557: from light fields, e.g. from plenoptic cameras
- G06T 7/586: from multiple light sources, e.g. photometric stereo
- G06V 10/14: Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of generating a depth map corresponding to input data includes providing light to a target object in a plurality of patterns that change over time, obtaining a plurality of captured images respectively corresponding to the plurality of patterns, by photographing the target object to which the light is provided, obtaining the input data by preprocessing the plurality of captured images and generating the depth map based on the input data.

## Claims

### Claim 1

A method of generating a depth map corresponding to input data, the method comprising:
providing light to a target object in a plurality of patterns that change over time;
obtaining a plurality of captured images respectively corresponding to the plurality of patterns, by photographing the target object to which the light is provided;
obtaining the input data by preprocessing the plurality of captured images; and
generating the depth map based on the input data.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the plurality of patterns comprises a light-emitting region to which the light is provided, and
wherein a position of the light-emitting region changes over time.

### Claim 3 (depends on 1)

The method of claim 1, wherein each of the plurality of patterns comprises a light-emitting region to which the light is provided, and
wherein an area of at least a partial region of the light-emitting region changes over time.

### Claim 4 (depends on 1)

The method of claim 1, wherein an illumination of the light in each of the plurality of patterns changes over time.

### Claim 5 (depends on 1)

The method of claim 1, further comprising obtaining a plurality of illumination images respectively corresponding to the plurality of patterns by photographing a light-emitting device configured to provide the light,
wherein the obtaining of the input data comprises obtaining the input data by preprocessing the plurality of captured images and the plurality of illumination images.

### Claim 6 (depends on 5)

The method of claim 5, further comprising obtaining characteristic information of the light-emitting device,
wherein the obtaining of the input data comprises obtaining the input data by preprocessing the plurality of captured images, the plurality of illumination images, and the characteristic information.

### Claim 7 (depends on 6)

The method of claim 6, wherein the characteristic information comprises information about a size of the light-emitting device.

### Claim 8 (depends on 6)

The method of claim 6, further comprising:
providing sub-light to the target object; and
obtaining sub-characteristic information of a sub-light-emitting device configured to provide the sub-light,
wherein the obtaining of the plurality of captured images comprises obtaining the plurality of captured images by photographing the target object to which the light and the sub-light are provided, and
wherein the obtaining of the input data comprises obtaining the input data by preprocessing the plurality of captured images, the plurality of illumination images, the characteristic information, and the sub-characteristic information.

### Claim 9 (depends on 1)

The method of claim 1, further comprising comparing an intensity of an ambient illumination of the target object with an intensity of a threshold illumination,
wherein the light is provided to the target object based on the intensity of the ambient illumination being less than or equal to the intensity of the threshold illumination, and
wherein the threshold illumination is a maximum illumination at which the plurality of captured images reflecting changes in the plurality of patterns that change over time are able to be obtained.

### Claim 10 (depends on 1)

The method of claim 1, wherein the generating of the depth map comprises generating the depth map by providing a depth map generation module with the input data, and
wherein the depth map generation module comprises an autoencoder.

### Claim 11

An electronic device for generating a depth map corresponding to input data, the electronic device comprising:
a light-emitting device configured to provide light to a target object in a plurality of patterns that change over time;
a measuring device configured to obtain a plurality of captured images respectively corresponding to the plurality of patterns by photographing the target object to which the light is provided;
a memory storing one or more instructions; and
at least one processor configured to execute the one or more instructions stored in the memory to
obtain the input data by preprocessing the plurality of captured images, and
generate the depth map based on the input data.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein each of the plurality of patterns comprises a light-emitting region to which the light is provided, and
wherein a position of the light-emitting region changes over time.

### Claim 13 (depends on 11)

The electronic device of claim 11, wherein each of the plurality of patterns comprises a light-emitting region to which the light is provided, and
wherein an area of at least a partial region of the light-emitting region changes over time.

### Claim 14 (depends on 11)

The electronic device of claim 11, wherein an illumination of the light in each of the plurality of patterns changes over time.

### Claim 15 (depends on 11)

The electronic device of claim 11, wherein the at least one processor is further configured to execute the one or more instructions to
receive a plurality of illumination images that are previously captured, the plurality of illumination images being obtained by photographing the light-emitting device, and
obtain the input data by preprocessing the plurality of captured images and the plurality of illumination images,
wherein the plurality of illumination images respectively correspond to the plurality of patterns.

### Claim 16 (depends on 15)

The electronic device of claim 15, wherein the at least one processor is further configured to execute the one or more instructions to
receive characteristic information of the light-emitting device, and
obtain the input data by preprocessing the plurality of captured images, the plurality of illumination images, and the characteristic information.

### Claim 17 (depends on 16)

The electronic device of claim 16, wherein the characteristic information comprises information about a size of the light-emitting device.

### Claim 18 (depends on 16)

The electronic device of claim 16, further comprising a sub-light-emitting device configured to provide sub-light to the target object,
wherein the measuring device is further configured to obtain the plurality of captured images by photographing the target object to which the light and the sub-light are provided, and
wherein the at least one processor is further configured to execute the one or more instructions to
receive sub-characteristic information of the sub-light-emitting device, and
obtain the input data by preprocessing the plurality of captured images, the plurality of illumination images, the characteristic information, and the sub-characteristic information.

### Claim 19 (depends on 11)

The electronic device of claim 11, wherein the at least one processor is further configured to execute the one or more instructions to generate the depth map by providing a depth map generation module with the input data, and
wherein the depth map generation module comprises an autoencoder.

### Claim 20

A non-transitory computer-readable recording medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
provide light toward a target object in a plurality of patterns that change over time;
obtain a plurality of captured images respectively corresponding to the plurality of patterns, by photographing the target object toward which the light is provided;
obtain input data by preprocessing the plurality of captured images; and
generate a depth map based on the input data.
