# US 10,762,689 B2 · Method and apparatus for selecting a surface in a light field, and corresponding computer program product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,762,689 B2
- Document ID: US10762689B2
- Status: Granted
- Kind: B2
- Application number: 16/170,608
- Filed: 2018-10-25
- Published: 2019-05-02
- Granted: 2020-09-01
- Earliest claimed: 2017-10-31
- Assignees: InterDigital CE Patent Holdings, SAS
- Applicants: Interdigital Ce Patent Holdings
- Inventors: Neus Sabater; Erik Reinhard; Matthieu Hog; Elena Garces
- Web page: https://patentgrep.com/patent/US10762689B2

## CPC

- G06T 11/60: Creating or editing images; Combining images with text
- G06T 15/20: Perspective computation
- G06T 15/40: Hidden part removal
- G06T 15/50: Lighting effects
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2200/21: involving computational photography
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10052: Images from lightfield camera
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Selecting at least one surface in a light field, such as a light field associated with multiple views corresponding to different view points of a same scene, may include displaying one of the multiple views as a reference view, selecting at least one surface in the reference view, identifying a set of pixels belonging to the selected surface, which are occluded in the reference view but visible in at least one of the multiple views, projecting the set of pixels into the reference view on the basis of a value of the pixels of the set in the view or views in which they are visible in order to form an upgraded reference view, displaying the upgraded reference view, and updating the selected surface in the upgraded reference view.

## Claims

### Claim 1

A method for selecting at least one surface in a light field, said light field being associated with multiple views corresponding to different view points of a same scene, wherein the method comprises:
selecting one of said multiple views as a reference view;
selecting at least one surface in said reference view;
identifying a set of pixels belonging to said at least one surface, which are occluded in said reference view but visible in at least one of said multiple views;
projecting said set of pixels into said reference view, on the basis of a value of said pixels of said set in said at least one of said multiple views in which they are visible, in order to form an upgraded reference view;
updating said selected surface in said upgraded reference view.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
detecting at least one foreground object at least partially occluding said at least one surface in said reference view;
hiding said at least one foreground object in said upgraded reference view.

### Claim 3 (depends on 2)

The method of claim 2, wherein hiding said at least one foreground object comprises applying a transparency effect to said object.

### Claim 4 (depends on 3)

The method of claim 3, further comprising editing said updated selected surface, and wherein said editing belongs to the group comprising:
changing color;
changing texture;
changing illumination;
scaling;
stretching.

### Claim 5 (depends on 4)

The method according to claim 4, wherein said reference view is a central view of said multiple views.

### Claim 6 (depends on 1)

The method of claim 1 further comprising:
displaying the reference view before selecting the at least one surface; and
displaying the upgraded reference view before updating the selected surface in the upgraded reference view.

### Claim 7

An apparatus for selecting at least one surface in a light field, said light field being associated with multiple views corresponding to different view points of a same scene, wherein the apparatus comprises:
one or more processors configured to
select one of said multiple views as a reference view;
select at least one surface in said reference view;
identify a set of pixels belonging to said at least one surface, which are occluded in said reference view but visible in at least one of said multiple views;
projecting said set of pixels into said reference view, on the basis of a value of said pixels of said set in said at least one of said multiple views in which they are visible, in order to form an upgraded reference view;
update said selected surface in said upgraded reference view.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the one or more processors being further configured to perform an edit of said updated selected surface, and wherein the edit belongs to the group comprising:
changing color;
changing texture;
changing illumination;
scaling;
stretching.

### Claim 9 (depends on 7)

The apparatus of claim 7, wherein the one or more processors being further configured to:
detect at least one foreground object at least partially occluding said at least one surface in said reference view;
hide said at least one foreground object in said upgraded reference view.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the one or more processors being further configured to hide said at least one foreground object comprises the one or more processors being configured to apply a transparency effect to said object.

### Claim 11 (depends on 7)

The apparatus of claim 7, wherein the one or more processors are further configured to
display the reference view before selecting the at least one surface; and
display the upgraded reference view before updating the selected surface in the upgraded reference view.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein said reference view is a central view of said multiple views.

### Claim 13

A non-transitory computer-readable medium storing program code instructions executable by a processor for selecting at least one surface in a light field, said light field being associated with multiple views corresponding to different view points of a same scene, and for performing operations comprising:
selecting one of said multiple views as a reference view;
selecting at least one surface in said reference view;
identifying a set of pixels belonging to said at least one surface, which are occluded in said reference view but visible in at least one of said multiple views;
projecting said set of pixels into said reference view, on the basis of a value of said pixels of said set in said at least one of said multiple views in which they are visible, in order to form an upgraded reference view;
updating said selected surface in said upgraded reference view.

### Claim 14 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the operations performed by the processor executing the stored program code instructions further comprise:
detecting at least one foreground object at least partially occluding said at least one surface in said reference view;
hiding said at least one foreground object in said upgraded reference view.

### Claim 15 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein the operation hiding said at least one foreground object performed by the processor executing the stored program code instructions comprises applying a transparency effect to said object.

### Claim 16 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the operations performed by the processor executing the stored program code instructions further comprise editing said updated selected surface, and wherein said editing belongs to the group comprising:
changing color;
changing texture;
changing illumination;
scaling;
stretching.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein said reference view is a central view of said multiple views.

### Claim 18 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the operations performed by the processor executing the stored program code instructions further comprise:
displaying the reference view before selecting the at least one surface in said reference view; and
displaying the upgraded reference view before updating the selected surface in the upgraded reference view.