# US 2019/0320164 A1 · Multi-pass add-on tool for coherent and complete view synthesis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320164 A1
- Document ID: US20190320164A1
- Status: Published application
- Kind: A1
- Application number: 16/456,197
- Filed: 2019-06-28
- Published: 2019-10-17
- Earliest claimed: 2019-03-19
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Basel Salahieh; Sumit Bhatia; Jill Boyce
- Web page: https://patentgrep.com/patent/US20190320164A1

## CPC

- H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/593: from stereo images
- H04N 13/156: Mixing image signals
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 13/344: with head-mounted left-right displays
- H04N 13/351: for displaying simultaneously
- H04N 19/597: specially adapted for multi-view video sequence encoding
- G06T 2207/20221: Image fusion; Image merging
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An embodiment of an image processor for immersive video includes technology to re-order patches from a plurality of views based on one or more of relative position and orientation related information for a desired synthesized view, select a set of views to be used in each view synthesis pass, perform two or more view synthesis passes for the synthesized view to provide two or more intermediate view synthesis results, and mask and merge the two or more intermediate view synthesis results to provide a final view synthesis result. Other embodiments are disclosed and claimed.

## Claims

### Claim 1

An electronic system, comprising:
memory to store one or more patches from a plurality of views;
a processor communicatively coupled to the memory; and
logic coupled to the processor, the logic to:
re-order the one or more patches from the plurality of views based on one or more of relative position and orientation related information for a desired synthesized view;
select a respective set of views to be used for each of two or more view synthesis passes;
perform the two or more view synthesis passes for the desired synthesized view using only patches belonging to the selected set of views for each pass to provide two or more intermediate view synthesis results; and
mask and merge the two or more intermediate view synthesis results to provide a final view synthesis result.

### Claim 2 (depends on 1)

The system of claim 1, wherein the logic is further to:
re-order the one or more patches from the plurality of views based on a distance between each view and the desired synthesized view.

### Claim 3 (depends on 2)

The system of claim 2, wherein the logic is further to:
determine overlapped scene coverage information between the plurality of views and the desired synthesized view based on the one or more of relative position and orientation related information; and
re-order the one or more patches from the plurality of views in an order from a relatively largest overlap to a relatively smallest overlap based on the determined overlapped scene coverage information between the plurality of views and the desired synthesized view.

### Claim 4 (depends on 1)

The system of claim 1, wherein the logic is further to:
perform two or more of the view synthesis passes in parallel with each other.

### Claim 5 (depends on 4)

The system of claim 4, wherein the logic is further to:
determine one or more of a number of passes and a number of views for the performed view synthesis based on one or more user-provided parameters.

### Claim 6 (depends on 1)

The system of claim 1, wherein the logic is further to:
generate a binary mask to identify missing information in an intermediate view.

### Claim 7 (depends on 6)

The system of claim 6, wherein the logic is further to:
merge two intermediate view synthesis results based on the binary mask such that missing information is taken from the higher pass intermediate synthesized view and coherent synthesis information is taken from the lower pass intermediate synthesized view; and
repeat the merge consecutively for each intermediate synthesized view result.

### Claim 8

A method of processing an image, comprising:
re-ordering one or more patches from a plurality of views based on one or more of relative position and orientation related information for a desired synthesized view;
selecting a respective set of views to be used for each of two or more view synthesis passes;
performing the two or more view synthesis passes for the desired synthesized view using only patches belonging to the selected set of views for each pass to provide two or more intermediate view synthesis results; and
masking and merging the two or more intermediate view synthesis results to provide a final view synthesis result.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
re-ordering the one or more patches from the plurality of views based on a distance between each view and the desired synthesized view.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
determining overlapped scene coverage information between the plurality of views and the desired synthesized view based on the one or more of relative position and orientation related information; and
re-ordering the one or more patches from the plurality of views in an order from a relatively largest overlap to a relatively smallest overlap based on the determined overlapped scene coverage information between the plurality of views and the desired synthesized view.

### Claim 11 (depends on 8)

The method of claim 8, further comprising:
performing two or more of the view synthesis passes in parallel with each other.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
determining one or more of a number of passes and a number of views for the performed view synthesis based on one or more user-provided parameters.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
generating a binary mask to identify missing information in an intermediate view.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
merging two intermediate view synthesis results based on the binary mask such that missing information is taken from the higher pass intermediate synthesized view and coherent synthesis information is taken from the lower pass intermediate synthesized view; and
repeating the merge consecutively for each intermediate synthesized view result.

### Claim 15

At least one machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to:
re-order one or more patches from a plurality of views based on one or more of relative position and orientation related information for a desired synthesized view;
select a respective set of views to be used for each of two or more view synthesis passes;
perform the two or more view synthesis passes for the desired synthesized view using only patches belonging to the selected set of views for each pass to provide two or more intermediate view synthesis results; and
mask and merge the two or more intermediate view synthesis results to provide a final view synthesis result.

### Claim 16 (depends on 15)

The machine readable medium of claim 15, comprising a plurality of further instructions that, in response to being executed on the computing device, cause the computing device to:
re-order the one or more patches from the plurality of views based on a distance between each view and the desired synthesized view.

### Claim 17 (depends on 16)

The machine readable medium of claim 16, comprising a plurality of further instructions that, in response to being executed on the computing device, cause the computing device to:
determine overlapped scene coverage information between the plurality of views and the desired synthesized view based on the one or more of relative position and orientation related information; and
re-order the one or more patches from the plurality of views in an order from a relatively largest overlap to a relatively smallest overlap based on the determined overlapped scene coverage information between the plurality of views and the desired synthesized view.

### Claim 18 (depends on 15)

The machine readable medium of claim 15, comprising a plurality of further instructions that, in response to being executed on the computing device, cause the computing device to:
perform two or more of the view synthesis passes in parallel with each other.

### Claim 19 (depends on 18)

The machine readable medium of claim 18, comprising a plurality of further instructions that, in response to being executed on the computing device, cause the computing device to:
determine one or more of a number of passes and a number of views for the performed view synthesis based on one or more user-provided parameters.

### Claim 20 (depends on 15)

The machine readable medium of claim 15, comprising a plurality of further instructions that, in response to being executed on the computing device, cause the computing device to:
generate a binary mask to identify missing information in an intermediate view.

### Claim 21 (depends on 20)

The machine readable medium of claim 20, comprising a plurality of further instructions that, in response to being executed on the computing device, cause the computing device to:
merge two intermediate view synthesis results based on the binary mask such that missing information is taken from the higher pass intermediate synthesized view and coherent synthesis information is taken from the lower pass intermediate synthesized view; and
repeat the merge consecutively for each intermediate synthesized view result.