# US 12,113,949 B2 · System and method for depth data coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,113,949 B2
- Document ID: US12113949B2
- Status: Granted
- Kind: B2
- Application number: 18/088,207
- Filed: 2022-12-23
- Published: 2024-06-27
- Granted: 2024-10-08
- Earliest claimed: 2022-12-23
- Assignees: Rovi Guides, Inc.
- Applicants: Rovi Guides, Inc.
- Inventors: Seppo Valli; Pekka Siltanen
- Web page: https://patentgrep.com/patent/US12113949B2

## CPC

- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 13/128: Adjusting depth or disparity
- H04N 13/194: Transmission of image signals
- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods for encoding/decoding a 3D image are provided. The system decomposes depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges, wherein each component depth map corresponds to a focal plane of a multiple focal plane (MFP) decomposition of the image data. The system generates a plurality of component depth map focal planes (CDMFPs) by combining each respective CDM with the depth map. The system scales data in each CDMFP by a respective scaling factor. The system generates for transmission a plurality of encoded scaled CDMFP data streams for the plurality of depth ranges, wherein each respective scaled CDMFP data stream is based at least in part on a respective scaled CDMFP.

## Claims

### Claim 1

A method comprising:
accessing image data that comprises a texture data and a depth map;
decomposing the depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges, wherein each component depth map corresponds to a focal plane of a multiple focal plane (MFP) decomposition of the image data;
generating a plurality of component depth map focal planes (CDMFPs) by combining each respective CDM with the depth map;
scaling data in each CDMFP, wherein each respective CDMFP is scaled by a respective scaling factor;
generating a plurality of encoded scaled CDMFP data streams for the plurality of depth ranges, wherein each respective scaled CDMFP data stream is based at least in part on a respective scaled CDMFP; and
transmitting the plurality of encoded scaled CDMFP data streams to a client device to cause the client device to:
reconstruct the depth map based on a plurality of decoded scaled CDMFPs; and
generate for display an image based on the reconstructed depth map.

### Claim 2 (depends on 1)

The method of claim 1, wherein the combining each respective CDM with the depth map comprises multiplying each respective CDM by the depth map.

### Claim 3 (depends on 1)

The method of claim 1, wherein each respective scaling factor is based on a maximum value in the respective CDMFP.

### Claim 4 (depends on 1)

The method of claim 1, wherein the plurality of encoded scaled CDMFP data streams are transmitted in parallel as part of a data container.

### Claim 5 (depends on 4)

The method of claim 4, wherein:
the plurality of encoded scaled CDMFP data streams are synchronized with the texture data; and
the transmitting the plurality of encoded CDM data streams to a client device comprises transmitting the plurality of encoded CDMFP data streams synchronized with the texture data to cause the client device to generate for display an image based on the reconstructed depth map and the synchronized texture data received as part of a data container.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
decomposing the depth map into a plurality of CDMs comprises applying a decomposition function to the depth map; and
the client is causes to reconstructing the depth map by summing de-scaled scaled CDMFPs.

### Claim 7 (depends on 6)

The method of claim 6, wherein the decomposition function is a set of tent functions.

### Claim 8 (depends on 6)

The method of claim 6, wherein the decomposition function is a set of sinusoid functions.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
identifying a key depth range by performing object detection on the image data;
selecting a key CDMFP of the plurality of CDMFPs that corresponds to the key depth range;
wherein the generating the plurality of encoded scaled CDMFP data streams comprises:
encoding the key CDMFP at higher bit rate than at least one other CDMFP of the plurality of CDMFPs.

### Claim 10 (depends on 9)

The method of claim 9, wherein the selection of the key CDMFP is performed during live streaming of the image data.

### Claim 11 (depends on 1)

The method of claim 1, wherein generating the plurality of encoded CDMFP data streams comprises:
separately pre-encoding each CDMFP at a plurality of bit rates.

### Claim 12 (depends on 11)

The method of claim 11, wherein transmitting the plurality of encoded CDMFP data streams comprises:
selecting a first bit rate of the plurality of bit rates for a first CDMFP of the plurality of CDMFPs; and
selecting a second bit rate of the plurality of bit rates for a second CDMFP of the plurality of CDMFPs;
transmitting data pre-encoded at the first bit rate for the first CDMFP; and
transmitting data pre-encoded at the second bit rate for the second CDMFP.

### Claim 13

A system comprising:
control circuitry configured to:
access image data that comprises a texture data and a depth map;
decompose the depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges, wherein each component depth map corresponds to a focal plane of a multiple focal plane (MFP) decomposition of the image data;
generate a plurality of component depth map focal planes (CDMFPs) by combining each respective CDM with the depth map;
scaling data in each CDMFP, wherein each respective CDMFP is scaled by a respective scaling factor;
generate a plurality of encoded scaled CDMFP data streams for the plurality of depth ranges, wherein each respective scaled CDMFP data stream is based at least in part on a respective scaled CDMFP; and
networking circuitry configured to:
transmit the plurality of encoded scaled CDMFP data streams to a client device to cause the client device to:
reconstruct the depth map based on a plurality of decoded scaled CDMFPs; and
generate for display an image based on the reconstructed depth map.

### Claim 14 (depends on 13)

The system of claim 13, wherein the control circuitry configured to combine each respective CDM with the depth map by multiplying each respective CDM by the depth map.

### Claim 15 (depends on 13)

The system of claim 13, wherein each respective scaling factor is based on a maximum value in the respective CDMFP.

### Claim 16 (depends on 13)

The system of claim 13, wherein the networking circuitry configured to transmit the plurality of encoded scaled CDMFP data streams in parallel as part of a data container.

### Claim 17 (depends on 16)

The system of claim 16, wherein:
the control circuitry is configured to synchronize the plurality of encoded scaled CDMFP data streams with the texture data; and
the networking circuitry is configured to transmit the plurality of encoded CDM data streams to a client device by transmitting the plurality of encoded CDMFP data streams synchronized with the texture data to cause the client device to generate for display an image based on the reconstructed depth map and the synchronized texture data received as part of a data container.

### Claim 18 (depends on 13)

The system of claim 13, wherein:
the control circuitry is configured to decompose the depth map into a plurality of CDMs by applying a decomposition function to the depth map; and
the client is caused to reconstructing the depth map by summing de-scaled scaled CDMFPs.

### Claim 19 (depends on 18)

The system of claim 18, wherein the decomposition function is a set of tent functions.

### Claim 20 (depends on 18)

The system of claim 18, wherein the decomposition function is a set of sinusoid functions.