# US 2020/0404247 A1 · System for and method of social interaction using user-selectable novel views

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404247 A1
- Document ID: US20200404247A1
- Status: Published application
- Kind: A1
- Application number: 16/888,008
- Filed: 2020-05-29
- Published: 2020-12-24
- Earliest claimed: 2014-04-30
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Oren Haimovitch-Yogev; Matteo Shapira; Aviv Shapira; Diego Prilusky; Yaniv Ben Zvi; Adi Gilat
- Web page: https://patentgrep.com/patent/US20200404247A1

## CPC

- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- G01B 11/245: using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence)
- G06T 7/11: Region-based segmentation
- G06T 7/50: Depth or shape recovery
- G06T 12/00: Tomographic reconstruction from projections
- G06T 15/10: Geometric effects
- G06T 15/205: {Image-based rendering}
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 13/194: Transmission of image signals
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- H04N 13/218: using spatial multiplexing
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/47202: {for requesting content on demand, e.g. video on demand}
- H04N 21/816: {involving special video data, e.g 3D video}
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T 2200/16: involving adaptation to the client's capabilities
- G06T 2207/20152: Watershed segmentation
- G06T 2210/22: Cropping
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system for social interaction using a photo-realistic novel view of an event includes a multi-view reconstruction system for developing transmission data of the event a plurality of client-side rendering devices, each rendering device receiving the transmission data from the multi-view reconstruction system and rendering the transmission data as the photo-realistic novel view. A method of social interaction using a photo-realistic novel view of an event includes transmitting by a server side transmission data of the event; receiving by a first user on a first rendering device the data transmission; selecting by the first user a path for rendering on the first rendering device at least on novel view; rendering by the first rendering device the at least one novel view; and saving by the user on the first rendering device novel view date for the at least one novel view.

## Claims

### Claim at position 1

1-7. (canceled)

### Claim 8

A client rendering device, comprising:
a central processing unit and a non-transitory computer-readable memory that stores instructions, when executed by the central processing unit, are capable of causing the client rendering device to perform:
obtaining image data that represents images captured from a plurality of cameras, and wherein each camera within the plurality of cameras is to capture a view field of an event space for an event different from what is captured by another camera within the plurality of cameras,
creating one or more environment images based on a novel view of the event, wherein the novel view is selected by a client through a user interface of the client rendering device, and wherein the novel view does not coincide with any view field of the plurality of cameras; and
rendering one or more images on the client rendering device based on the environment images and the novel view.

### Claim 9 (depends on 8)

The client rendering device of claim 8, wherein the rendered one or more images are to be stored in a social network database.

### Claim 10 (depends on 8)

The client rendering device of claim 8, wherein rendering the one or more images on the rendering device comprises rendering background data of the one or more images using a first rendering technique and rendering foreground data of the one or more images using a second rendering technique.

### Claim 11 (depends on 8)

The client rendering device of claim 8, wherein the instructions, when executed by the central processing unit, capable of causing the client rendering device to further perform:
uncompressing a sequence of image data received from a server to obtain the image data that represents images captured from a plurality of cameras.

### Claim 12 (depends on 8)

The client rendering device of claim 8, wherein creating the environment image comprises blending a set of environment images by weighting an angular distance of a virtual camera corresponding to the novel view from cameras that captured view fields of the event space.

### Claim 13 (depends on 8)

The client rendering device of claim 8, wherein rendering the one or more images on the rendering device based on the environment image and the novel view comprises:
overlaying luma differences on the environment images to create one or more of shadows, highlights, and brightness difference on background of the rendered images.

### Claim 14 (depends on 8)

The client rendering device of claim 8, wherein the instructions, when executed by the central processing unit, capable of causing the client rendering device to further perform:
sharing the rendered one or more images with one or more other users through one or more of direct text messaging, wall postings, uploading content to the internet, and broadfield broadcasting.

### Claim 15

A method to be implemented in a client rendering device, comprising:
obtaining image data that represents images captured from a plurality of cameras, and wherein each camera within the plurality of cameras is to capture a view field of an event space for an event different from what is captured by another camera within the plurality of cameras,
creating one or more environment images based on a novel view of the event, wherein the novel view is selected by a client through a user interface of the client rendering device, and wherein the novel view does not coincide with any view field of the plurality of cameras; and
rendering one or more images on the client rendering device based on the environment images and the novel view.

### Claim 16 (depends on 15)

The method of claim 15, wherein the rendered one or more images are to be stored in a social network database.

### Claim 17 (depends on 15)

The method of claim 15, wherein rendering the one or more images on the rendering device comprises rendering background data of the one or more images using a first rendering technique and rendering foreground data of the one or more images using a second rendering technique.

### Claim 18 (depends on 15)

The method of claim 15, wherein the method further comprises:
uncompressing a sequence of image data received from a server to obtain the image data that represents images captured from a plurality of cameras.

### Claim 19 (depends on 15)

The method of claim 15, wherein creating the environment image comprises blending a set of environment images by weighting an angular distance of a virtual camera corresponding to the novel view from cameras that captured view fields of the event space.

### Claim 20 (depends on 15)

The method of claim 15, wherein rendering the one or more images on the rendering device based on the environment image and the novel view comprises:
overlaying luma differences on the environment images to create one or more of shadows, highlights, and brightness difference on background of the rendered images.

### Claim 21 (depends on 15)

The method of claim 15, wherein the method further comprises:
sharing the rendered one or more images with one or more other users through one or more of direct text messaging, wall postings, uploading content to the internet, and broadfield broadcasting.

### Claim 22

A non-transitory computer-readable storage medium that stores instructions, when executed by a central processing unit, are capable of causing a client rendering device to perform:
obtaining image data that represents images captured from a plurality of cameras, and wherein each camera within the plurality of cameras is to capture a view field of an event space for an event different from what is captured by another camera within the plurality of cameras,
creating one or more environment images based on a novel view of the event, wherein the novel view is selected by a client through a user interface of the client rendering device, and wherein the novel view does not coincide with any view field of the plurality of cameras; and
rendering one or more images on the client rendering device based on the environment images and the novel view.

### Claim 23 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein the rendered one or more images are to be stored in a social network database.

### Claim 24 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein rendering the one or more images on the rendering device comprises rendering background data of the one or more images using a first rendering technique and rendering foreground data of the one or more images using a second rendering technique.

### Claim 25 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein the instructions, when executed by the central processing unit, capable of causing the client rendering device to further perform:
uncompressing a sequence of image data received from a server to obtain the image data that represents images captured from a plurality of cameras.

### Claim 26 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein creating the environment image comprises blending a set of environment images by weighting an angular distance of a virtual camera corresponding to the novel view from cameras that captured view fields of the event space.

### Claim 27 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein rendering the one or more images on the rendering device based on the environment image and the novel view comprises:
overlaying luma differences on the environment images to create one or more of shadows, highlights, and brightness difference on background of the rendered images.
