# US 2013/0156294 A1 · Depth map generation based on soft classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0156294 A1
- Document ID: US20130156294A1
- Status: Published application
- Kind: A1
- Application number: 13/327,733
- Filed: 2011-12-15
- Published: 2013-06-20
- Earliest claimed: 2011-12-15
- Assignees: Sony Corporation
- Inventors: JIANING WEI
- Web page: https://patentgrep.com/patent/US20130156294A1

## CPC

- G06T 7/50: Depth or shape recovery
- G06F 18/254: {of classification results, e.g. of results related to same input data}
- G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data}
- G06V 20/35: {Categorising the entire scene, e.g. birthday party or wedding scene}
- G06V 20/36: {Indoor scenes}
- G06V 20/38: {Outdoor scenes}
- G06T 2207/20076: Probabilistic image processing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for generating a depth map for a 2D image and video includes receiving the 2D image and video; defining a plurality of object classes; analyzing content of the received 2D image and video; calculating probabilities that the received 2D image belongs to the object classes; and determining a final depth map based on a result of the analyzed content and the calculated probabilities for the object classes.

## Claims

### Claim 1

A computer-implemented method for generating a depth map for a 2D image, the method comprising:
receiving the 2D image;
defining a plurality of object classes;
analyzing content of the received 2D image;
calculating probabilities that the received 2D image belongs to the object classes; and
determining a final depth map based on a result of the analyzed content and the calculated probabilities for the object classes.

### Claim 2 (depends on 1)

The method of claim 1, wherein analyzing content comprises:
acquiring a set of visual features from the received 2D image; and
comparing the received 2D image with the object classes based on the acquired set of visual features.

### Claim 3 (depends on 1)

The method of claim 1, wherein analyzing content further comprises:
assigning preliminary depth maps to the plurality of object classes, or computing preliminary depth maps for the plurality of object classes based on features in the received 2D image and corresponding object class properties.

### Claim 4 (depends on 3)

The method of claim 3, wherein determining a final depth map comprises:
computing the final depth map as a weighted sum of the assigned or computed preliminary depth maps and the calculated probabilities for the object classes.

### Claim 5 (depends on 4)

The method of claim 4, wherein computing the final depth map comprises:
calculating the products of the calculated probabilities and the preliminary depth maps for each of the object classes; and
summing the products for each of the object classes.

### Claim 6 (depends on 1)

The method of claim 1, wherein the object classes comprise:
a landscape class;
a city class;
an indoor class; and
a portrait class.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
applying an infinite impulse response filter to smooth the calculated probabilities for the object classes if the received 2D image is a fast action scene.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
if there are exactly two object classes in the plurality of object classes:
performing a single stage two-class classification of the received 2D image; and
calculating the probabilities for the two object classes in a single stage.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
if there are more than two object classes in the plurality of object classes:
performing a multi-stage two-class classification of the received 2D image;
calculating the probabilities for a first two object classes in a first stage;
calculating the probabilities for other object classes calculated in subsequent stages; and
combining the calculated probabilities to determine the probabilities that the received 2D image belongs to each of a plurality of combinations of object classes from the different stages.

### Claim 10 (depends on 9)

The method of claim 9, wherein performing a multi-stage two-class classification of the received 2D image comprises:
performing a two-stage two-class classification by:
executing a first stage classification by:
calculating the probability that the received 2D image belongs to a first object class; and
calculating the probability that the received 2D image belongs to a second object class;
executing a second stage classification by:
calculating the probability that the received 2D image belongs to a third object class; and
calculating the probability that the received 2D image belongs to a fourth class; and
combining the probabilities calculated in the first and second stage classifications.

### Claim 11 (depends on 10)

The method of claim 10, wherein combining the probabilities calculated in the first and second stage classifications comprises:
calculating, in the first and second stage classifications, the probability that the received 2D image belongs to each of a plurality of combinations of object classes.

### Claim 12 (depends on 11)

The method of claim 11, wherein calculating the probability that the received 2D image belongs to each of a plurality of combinations of object classes comprises:
determining the product of the calculated probability for each object class in each of the combinations of object classes.

### Claim 13 (depends on 1)

The method of claim 1, comprising receiving a plurality of 2D images.

### Claim 14

An apparatus for generating a depth map for a received 2D image, the apparatus comprising:
an image content analyzer to analyze content of the received 2D image;
a class probability calculator to calculate the probability that the received 2D image belongs to each of a plurality of object classes; and
a final depth map generator to determine a final depth map based on a result of the analyzed content and the calculated probabilities for the object classes.

### Claim 15 (depends on 14)

The apparatus of claim 14, further comprising:
an image database to store configuration information pertaining to the object classes for analyzing content of the received 2D image.

### Claim 16 (depends on 15)

The apparatus of claim 15, further comprising:
an image database to store configuration information pertaining to properties corresponding to the object classes.

### Claim 17 (depends on 14)

The image content analyzer of claim 14, further comprising:
a depth map generator to assign preliminary depth maps to the object classes, or to compute preliminary depth maps for the object classes using configuration information stored in the image database.

### Claim 18 (depends on 14)

The apparatus of claim 14, further comprising:
an infinite impulse response filter to smooth the calculated probabilities for the object classes if the received 2D image is a fast action scene.

### Claim 19

A non-transitory computer-readable medium storing instructions that, when executed, causes a computer to perform a method for generating a depth map for a 2D image, the method comprising:
receiving the 2D image;
defining a plurality of object classes;
analyzing content of the received 2D image;
calculating probabilities that the received 2D image belongs to the object classes; and
determining a final depth map based on a result of the analyzed content and the calculated probabilities for the object classes.
