# US 12,192,439 B2 · Method and device for processing image content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,192,439 B2
- Document ID: US12192439B2
- Status: Granted
- Kind: B2
- Application number: 17/800,119
- Filed: 2021-02-19
- Published: 2023-03-09
- Granted: 2025-01-07
- Earliest claimed: 2020-02-19
- Assignees: InterDigital VC Holdings, Inc.
- Applicants: InterDigital VC Holdings, Inc.
- Inventors: Benoit Vandame; Didier Doyen; Guillaume Boisson
- Web page: https://patentgrep.com/patent/US12192439B2

## CPC

- H04N 13/395: with depth sampling, i.e. the volume being constructed from a stack or sequence of two-dimensional [2D] image planes
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/11: Region-based segmentation
- G06T 7/90: Determination of colour characteristics
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06T 2200/21: involving computational photography
- G06T 2207/10024: Color image
- G06T 2207/10052: Images from lightfield camera
- G06T 2207/20221: Image fusion; Image merging
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method and system are provided for processing image content. In one embodiment the method comprises receiving a plurality of captured contents showing same scene as captured by one or more cameras having a different focal length and depth maps and generating a consensus cube by obtaining depth map estimations from said received contents. The visibility of different objects in then analysed to create a soft visibility cube that provides visibility information about each content. A color cube is then generated by using information from the consensus and soft visibility cube. The color cube is then used to combine different received contents and generate a single image for the plurality of contents received.

## Claims

### Claim 1

A method comprising:
obtaining a virtual color cube representing a volumetric scene from the point of view of a virtual camera, the virtual color cube comprising slices of the volumetric scene at different depths, each slice encoding a 2D array of pixels, wherein each pixel in each slice comprises a color specification and an alpha value;
partitioning each slice of the virtual color cube into tiles, each tile comprising a rectangular subset of the pixels of the slice; and
constructing an atlas of significant tiles, wherein constructing the atlas further comprises storing the pixels of an excluded tile in a residual image at a position corresponding to the position occupied by the tile in its original slice of the virtual color cube.

### Claim 2 (depends on 1)

The method of claim 1, further comprising for each tile in each slice of the virtual color cube, comparing the alpha values associated with the pixels of the tile to a threshold alpha value.

### Claim 3 (depends on 2)

The method of claim 2, further comprising comparing and then determining based on said comparison when each tile is to be included or excluded in the atlas of significant tiles.

### Claim 4 (depends on 3)

The method of claim 3, wherein constructing the atlas further comprises:
when processing a subsequent slice of the virtual color cube, adding the stored pixels of the excluded tile to the pixels of a corresponding tile of the subsequent slice to produce modified pixels, and
using the modified pixels when performing the comparing, determining, including and excluding steps of constructing the atlas for the corresponding tile.

### Claim 5 (depends on 1)

The method of claim 1, wherein constructing the atlas further comprises processing the slices of the virtual color cube in order of their depth values.

### Claim 6 (depends on 1)

The method of claim 1, wherein the color specification comprises R, G, and B color values.

### Claim 7 (depends on 1)

The method of claim 1, wherein the alpha value for a pixel reflects a confidence that the pixel represents a portion of the volumetric scene accurately positioned at the depth corresponding to the slice in which the pixel resides.

### Claim 8 (depends on 1)

The method of claim 1, wherein the alpha value for a pixel represents a probability.

### Claim 9 (depends on 1)

The method of claim 1, wherein the alpha value for a pixel is a value in the range from 0 to 1.

### Claim 10 (depends on 1)

The method of claim 1, wherein the alpha value for a pixel reflects the degree to which multiple cameras capturing the scene agree that a surface of an object exists at the pixel position within the slice.

### Claim 11 (depends on 1)

The method of claim 1, wherein each slice of the virtual color cube is partitioned using the same pattern of tiles.

### Claim 12

An apparatus comprising:
a processor configured to:
obtain a virtual color cube representing a volumetric scene from the point of view of a virtual camera, the virtual color cube comprising slices of the volumetric scene at different depths, each slice encoding a 2D array of pixels, wherein each pixel in each slice comprises a color specification and an alpha value;
partition each slice of the virtual color cube into tiles, each tile comprising a rectangular subset of the pixels of the slice; and
construct an atlas of significant tiles, comprising storing the pixels of an excluded tile in a residual image at a position corresponding to the position occupied by the tile in its original slice of the virtual color cube.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the processor is further configured to, for each tile in each slice of the virtual color cube, compare the alpha values associated with the pixels of the tile to a threshold alpha value.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the processor is further configured to compare and determine based on said comparison when each tile is to be included or excluded in the atlas of significant tiles.

### Claim 15 (depends on 13)

The apparatus of claim 13, wherein the processor is further configured to:
when processing a subsequent slice of the virtual color cube, add the stored pixels of the excluded tile to the pixels of a corresponding tile of the subsequent slice to produce modified pixels; and
use the modified pixels when performing the comparing, determining, including and excluding steps of constructing the atlas for the corresponding tile.

### Claim 16 (depends on 12)

The apparatus of claim 12, wherein the processor is further configured to:
process the slices of the virtual color cube in order of their depth values.

### Claim 17 (depends on 12)

The apparatus of claim 12, wherein the color specification comprises R, G, and B color values.

### Claim 18 (depends on 12)

The apparatus of claim 12, wherein the alpha value for a pixel reflects a confidence that the pixel represents a portion of the volumetric scene accurately positioned at the depth corresponding to the slice in which the pixel resides.

### Claim 19 (depends on 12)

The apparatus of claim 12, wherein the alpha value for a pixel reflects the degree to which multiple cameras capturing the scene agree that a surface of an object exists at the pixel position within the slice.

### Claim 20 (depends on 12)

The apparatus of claim 12, wherein each slice of the virtual color cube is partitioned using the same pattern of tiles.