# US 2024/0203139 A1 · Systems and methods for depth map sampling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203139 A1
- Document ID: US20240203139A1
- Status: Published application
- Kind: A1
- Application number: 18/408,031
- Filed: 2024-01-09
- Published: 2024-06-20
- Earliest claimed: 2015-08-24
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Albrecht Johannes LINDNER; Volodimir SLOBODYANYUK; Stephen Michael VERRALL; Kalin Mitkov ATANASSOV
- Web page: https://patentgrep.com/patent/US20240203139A1

## CPC

- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G01S 7/4972: {Alignment of sensor}
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/89: for mapping or imaging
- G06T 7/50: Depth or shape recovery
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- H04N 13/261: with monoscopic-to-stereoscopic image conversion
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An electronic imaging device and method for image capture are described. The imaging device includes a camera configured to obtain image information of a scene and that may be focused on a region of interest in the scene. The imaging device also includes a LIDAR unit configured to obtain depth information of at least a portion of the scene at specified scan locations of the scene. The imaging device is configured to detect an object in the scene and provides specified scan locations to the LIDAR unit. The camera is configured to capture an image with an adjusted focus based on depth information, obtained by the LIDAR unit, associated with the detected object.

## Claims

### Claim 1

(canceled)

### Claim 2

An imaging device, comprising:
a depth sensor configured to obtain depth information of at least a portion of a scene; and
a processor configured to:
detect an object in the scene using image information of the scene and first depth information obtained from the depth sensor;
provide one or more instructions to the depth sensor to obtain second depth information at a plurality of specified scan locations, wherein the plurality of specified scan locations are characterized by a first scan point density for a first portion of the scene associated with the object and a second scan point density for a second portion of the scene; and
output the second depth information.

### Claim 3 (depends on 2)

The imaging device of claim 2, wherein the depth sensor is a light-detection and ranging (LIDAR) unit configured to capture LIDAR data.

### Claim 4 (depends on 2)

The imaging device of claim 2, wherein the processor is configured to:
generate a first image segmentation of the image information associated with the object; and
generate a second image segmentation of the image information associated with a portion of the scene not associated with the object.

### Claim 5 (depends on 4)

The imaging device of claim 4, wherein one or more scan locations of the first portion of the scene are based on the first image segmentation, and wherein one or more scan locations of the second portion of the scene are based on the second image segmentation.

### Claim 6 (depends on 2)

The imaging device of claim 2, wherein the first scan point density is greater than the second scan point density.

### Claim 7 (depends on 2)

The imaging device of claim 2, wherein the object is a person or a face of the person.

### Claim 8 (depends on 7)

The imaging device of claim 7, wherein the processor is configured:
detect the face using facial recognition.

### Claim 9 (depends on 2)

The imaging device of claim 2, wherein the processor is configured to:
generate a segmentation of the image information, wherein the first scan point density is based on a similarity of image information of neighboring image segments within the object and a similarity of first depth information of neighboring image segments within the object.

### Claim 10 (depends on 2)

The imaging device of claim 2, further comprising a user interface configured to receive user input indicating a selection of the object, wherein the processor is configured to detect the object based on the user input indicating the selection of the object.

### Claim 11 (depends on 2)

The imaging device of claim 2, wherein the processor is configured to:
generate a depth map for at least a portion of the scene based on the second depth information.

### Claim 12 (depends on 11)

The imaging device of claim 11, wherein the processor is configured to:
generate a three-dimensional model of the object based on the depth map.

### Claim 13 (depends on 11)

The imaging device of claim 11, wherein the processor is configured to:
generate an image segmentation of the image information associated with the object; and
generate a refined depth map based on the image segmentation.

### Claim 14 (depends on 2)

The imaging device of claim 2, wherein the processor is configured to:
determine, based on the second depth information, a focus amount for a camera configured to capture the image information.

### Claim 15 (depends on 2)

The imaging device of claim 2, wherein the processor is configured to:
detect one or more objects in the scene based on the second depth information.

### Claim 16 (depends on 15)

The imaging device of claim 15, wherein the processor is configured to:
output an indication of the detected one or more objects in the scene for controlling a vehicle.

### Claim 17

A method for image capture, the method comprising:
detecting an object in a scene using image information of the scene and first depth information obtained from a depth sensor;
outputting one or more instructions to the depth sensor to obtain second depth information at a plurality of specified scan locations, wherein the plurality of specified scan locations are characterized by a first scan point density for a first portion of the scene associated with the object and a second scan point density for a second portion of the scene; and
outputting the second depth information.

### Claim 18 (depends on 17)

The method of claim 17, wherein the depth sensor is a light-detection and ranging (LIDAR) unit configured to capture LIDAR data.

### Claim 19 (depends on 17)

The method of claim 17, further comprising:
generating a first image segmentation of the image information associated with the object; and
generating a second image segmentation of the image information associated with a portion of the scene not associated with the object.

### Claim 20 (depends on 19)

The method of claim 19, wherein one or more scan locations of the first portion of the scene are based on the first image segmentation, and wherein one or more scan locations of the second portion of the scene are based on the second image segmentation.

### Claim 21 (depends on 17)

The method of claim 17, wherein the first scan point density is greater than the second scan point density.

### Claim 22 (depends on 17)

The method of claim 17, wherein the object is a person or a face of the person.

### Claim 23 (depends on 22)

The method of claim 22, further comprising:
detecting the face using facial recognition.

### Claim 24 (depends on 17)

The method of claim 17, further comprising:
generating a segmentation of the image information, wherein the first scan point density is based on a similarity of image information of neighboring image segments within the object and a similarity of first depth information of neighboring image segments within the object.

### Claim 25 (depends on 17)

The method of claim 17, further comprising:
receiving, via a user interface, user input indicating a selection of the object; and
detecting the object based on the user input indicating the selection of the object.

### Claim 26 (depends on 17)

The method of claim 17, further comprising:
generating a depth map for at least a portion of the scene based on the second depth information.

### Claim 27 (depends on 26)

The method of claim 26, further comprising:
generating a three-dimensional model of the object based on the depth map.

### Claim 28 (depends on 26)

The method of claim 26, further comprising:
generating an image segmentation of the image information associated with the object; and
generating a refined depth map based on the image segmentation.

### Claim 29 (depends on 17)

The method of claim 17, further comprising:
determining, based on the second depth information, a focus amount for a camera configured to capture the image information.

### Claim 30 (depends on 17)

The method of claim 17, further comprising:
detecting one or more objects in the scene based on the second depth information.

### Claim 31 (depends on 30)

The method of claim 30, further comprising:
outputting an indication of the detected one or more objects in the scene for controlling a vehicle.
