# US 2024/0005541 A1 · Image depth prediction method and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005541 A1
- Document ID: US20240005541A1
- Status: Published application
- Kind: A1
- Application number: 18/468,293
- Filed: 2023-09-15
- Published: 2024-01-04
- Earliest claimed: 2021-03-16
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Yue GONG; Xinjun MA; Jianan HE; LEI CHEN; Fan YU
- Web page: https://patentgrep.com/patent/US20240005541A1

## CPC

- G06T 7/593: from stereo images
- G06T 7/596: {from three or more stereo images}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/62: of area, perimeter, diameter or volume
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

This application relates to the field of image processing technologies, and in some embodiments, to an image depth prediction method and an electronic device. The method includes: obtaining a primary view and a first secondary view, where the primary view includes a first pixel, the first pixel corresponds to a second pixel at a first assumed depth, and the second pixel is located on the first secondary view; updating feature information of the second pixel based on the feature information of the second pixel and feature information of at least one third pixel, the fourth pixel is a pixel that corresponds to the first pixel at a second assumed depth and that is on the first secondary view; and obtaining a probability of the first assumed depth based on the feature information of the first pixel and the updated feature information of the second pixel.

## Claims

### Claim 1

An image depth prediction method, comprising:
obtaining a primary view and a first secondary view, wherein the primary view and the first secondary view are images photographed by a camera facing a first object at different angles, the primary view comprises a first pixel corresponding to a second pixel at a first assumed depth, and the second pixel is located on the first secondary view;
updating feature information of the second pixel based on the feature information of the second pixel and feature information of at least one third pixel, wherein the at least one third pixel is located on a connection line between the second pixel and a fourth pixel, the fourth pixel is a pixel that corresponds to the first pixel at a second assumed depth and that is on the first secondary view, the second assumed depth and the first assumed depth are adjacent in preset D assumed depths, and D is a positive integer greater than 1; and
obtaining a probability of the first assumed depth based on feature information of the first pixel and the updated feature information of the second pixel, wherein the probability of the first assumed depth indicates a possibility that the first assumed depth is a real depth of the first pixel.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the method further comprises:
updating feature information of the fourth pixel based on the feature information of the fourth pixel and feature information of at least one fifth pixel located on a connection line between the second pixel and the fourth pixel; and
obtaining a probability of the second assumed depth based on the feature information of the first pixel and the updated feature information of the fourth pixel, wherein the probability of the second assumed depth indicates a possibility that the second assumed depth is the real depth of the first pixel.

### Claim 3 (depends on 2)

The method according to claim 2, wherein
the at least one fifth pixel and the at least one third pixel are different from each other, and the at least one fifth pixel is closer to the fourth pixel than the at least one third pixel; or
a pixel included in the at least one fifth pixel but not included in the at least one third pixel is closer to the fourth pixel than a pixel included in the at least one third pixel but not included in the at least one fifth pixel.

### Claim 4 (depends on 2)

The method according to claim 2, wherein a union of the at least one fifth pixel and the at least one third pixel comprises all pixels on the connection line between the second pixel and the fourth pixel.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the method further comprises:
scaling down a size of a value range corresponding to the D assumed depths to obtain a size of a first range, wherein the first range is used as a value range of an assumed depth during depth re-prediction of the first pixel.

### Claim 6 (depends on 5)

The method according to claim 5, wherein a scale-down amplitude of the value range corresponding to the D assumed depths is negatively correlated to an entropy value of the first pixel at the D assumed depths.

### Claim 7 (depends on 4)

The method according to claim 4, wherein the method further comprises:
scaling down a size of a value range corresponding to D′ assumed depths to obtain a size of a second range representing a value range of an assumed depth during depth re-prediction of a sixth pixel, a scale-down amplitude of the value range corresponding to the D′ assumed depths is equal to a scale-down amplitude of a value range corresponding to the D assumed depths, the sixth pixel is a pixel in the primary view and is different from the first pixel, and D′ is a positive integer greater than 1.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the scale-down amplitude of the value range corresponding to the D assumed depths is negatively correlated to an entropy value of the first pixel at the D assumed depths and is negatively correlated to an entropy value of the sixth pixel at the D′ assumed depths.

### Claim 9

An electronic device, wherein the electronic device comprises:
at least one processor; and
at least one processor memory coupled to the at least one processor to store program instructions, which when executed by the at least one processor, cause the at least one processor to:
obtain a primary view and a first secondary view, wherein the primary view and the first secondary view are images photographed by a camera facing a first object at different angles, the primary view comprises a first pixel corresponding to a second pixel at a first assumed depth, and the second pixel is located on the first secondary view;
update feature information of the second pixel based on the feature information of the second pixel and feature information of at least one third pixel, wherein the at least one third pixel is located on a connection line between the second pixel and a fourth pixel, the fourth pixel is a pixel that corresponds to the first pixel at a second assumed depth and that is on the first secondary view, the second assumed depth and the first assumed depth are adjacent in preset D assumed depths, and D is a positive integer greater than 1; and
obtain a probability of the first assumed depth based on feature information of the first pixel and the updated feature information of the second pixel, wherein the probability of the first assumed depth indicates a possibility that the first assumed depth is a real depth of the first pixel.

### Claim 10 (depends on 9)

The electronic device according to claim 9, wherein the program instructions when executed further cause the at least one processor to:
update feature information of the fourth pixel based on the feature information of the fourth pixel and feature information of at least one fifth pixel located on a connection line between the second pixel and the fourth pixel; and
obtain a probability of the second assumed depth based on the feature information of the first pixel and the updated feature information of the fourth pixel, wherein the probability of the second assumed depth indicates a possibility that the second assumed depth is the real depth of the first pixel.

### Claim 11 (depends on 10)

The electronic device according to claim 10, wherein
the at least one fifth pixel and the at least one third pixel are different from each other, and the at least one fifth pixel is closer to the fourth pixel than the at least one third pixel; or
a pixel included in the at least one fifth pixel but not included in the at least one third pixel is closer to the fourth pixel than a pixel included in the at least one third pixel but not included in the at least one fifth pixel.

### Claim 12 (depends on 10)

The electronic device according to claim 10, wherein a union of the at least one fifth pixel and the at least one third pixel comprises all pixels on the connection line between the second pixel and the fourth pixel.

### Claim 13 (depends on 9)

The electronic device according to claim 9, wherein the program instructions when executed further cause the at least one processor to:
scale down a size of a value range corresponding to the D assumed depths to obtain a size of a first range, wherein the first range is used as a value range of an assumed depth during depth re-prediction of the first pixel.

### Claim 14 (depends on 12)

The electronic device according to claim 12, wherein a scale-down amplitude of the value range corresponding to the D assumed depths is negatively correlated to an entropy value of the first pixel at the D assumed depths.

### Claim 15 (depends on 12)

The electronic device according to claim 12, wherein the program instructions when executed further cause the at least one processor to:
scaling down a size of a value range corresponding to D′ assumed depths to obtain a size of a second range, wherein the second range is used as a value range of an assumed depth during depth re-prediction of a sixth pixel, a scale-down amplitude of the value range corresponding to the D′ assumed depths is equal to a scale-down amplitude of a value range corresponding to the D assumed depths, the sixth pixel is a pixel in the primary view and is different from the first pixel, and D′ is a positive integer greater than 1.

### Claim 16 (depends on 15)

The electronic device according to claim 15, wherein the scale-down amplitude of the value range corresponding to the D assumed depths is negatively correlated to an entropy value of the first pixel at the D assumed depths and is negatively correlated to an entropy value of the sixth pixel at the D′ assumed depths.

### Claim 17

Anon-transitory computer-readable storage medium, storing one or more instructions that, when executed by at least one processor, cause the at least one processor to:
obtain a primary view and a first secondary view, wherein the primary view and the first secondary view are images photographed by a camera facing a first object at different angles, the primary view comprises a first pixel corresponding to a second pixel at a first assumed depth, and the second pixel is located on the first secondary view;
update feature information of the second pixel based on the feature information of the second pixel and feature information of at least one third pixel, wherein the at least one third pixel is located on a connection line between the second pixel and a fourth pixel, the fourth pixel is a pixel that corresponds to the first pixel at a second assumed depth and that is on the first secondary view, the second assumed depth and the first assumed depth are adjacent in preset D assumed depths, and D is a positive integer greater than 1; and
obtain a probability of the first assumed depth based on feature information of the first pixel and the updated feature information of the second pixel, wherein the probability of the first assumed depth indicates a possibility that the first assumed depth is a real depth of the first pixel.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein the instructions when executed further cause the at least one processor to:
update feature information of the fourth pixel based on the feature information of the fourth pixel and feature information of at least one fifth pixel located on a connection line between the second pixel and the fourth pixel; and
obtain a probability of the second assumed depth based on the feature information of the first pixel and the updated feature information of the fourth pixel, wherein the probability of the second assumed depth indicates a possibility that the second assumed depth is the real depth of the first pixel.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18, wherein
the at least one fifth pixel and the at least one third pixel are different from each other, and the at least one fifth pixel is closer to the fourth pixel than the at least one third pixel; or
a pixel included in the at least one fifth pixel but not included in the at least one third pixel is closer to the fourth pixel than a pixel included in the at least one third pixel but not included in the at least one fifth pixel.

### Claim 20 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18, wherein a union of the at least one fifth pixel and the at least one third pixel comprises all pixels on the connection line between the second pixel and the fourth pixel.
