# US 2020/0349187 A1 · Method and apparatus for data retrieval in a lightfield database

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0349187 A1
- Document ID: US20200349187A1
- Status: Published application
- Kind: A1
- Application number: 16/934,636
- Filed: 2020-07-21
- Published: 2020-11-05
- Earliest claimed: 2015-06-17
- Assignees: InterDigital CE Patent Holdings, SAS
- Applicants: Interdigital Ce Patent Holdings
- Inventors: Jurgen STAUDER; Neus SABATER; Cedric THEBAULT
- Web page: https://patentgrep.com/patent/US20200349187A1

## CPC

- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/583: using metadata automatically derived from the content
- G06F 16/5854: {using shape and object relationship}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06T 7/60: Analysis of geometric attributes
- G06T 2207/10052: Images from lightfield camera
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method and an apparatus for data retrieval in a lightfield database are described. An acquiring unit of the apparatus acquires a sample data. An operation unit determines a plurality of two-dimensional (2D) images from each 4D lightfield data of the lightfield database, and match the sample data with the determined 2D images according to geometrical parameters of the sample data and geometrical parameters of the 2D images. One of the 2D images is selected as a target image based on the matching result, and the 4D lightfield from which the target image is determined is retrieved.

## Claims

### Claim 1

A method for querying a database of 4D Lightfield Data objects, comprising:
determining to query the database using a search query image;
generating, for each 4D Lightfield Data object in the database, multiple 2D images which differ from one another in at least one geometrical parameter;
performing a matching operation between the search query image and the generated 2D images to determine a target image, the target image being the generated 2D image which most closely matches the search query image according to a metric;
determining a target 4D Lightfield Data object, as the 4D Lightfield Data object from which the target image was generated; and
retrieving the target 4D Lightfield Data object in response to the query.

### Claim 2 (depends on 1)

The method of claim 1, wherein the search query image is determined based on user input.

### Claim 3 (depends on 1)

The method of claim 1, wherein the search query image is a representative image generated from a 4D Lightfield Data query object.

### Claim 4 (depends on 1)

The method of claim 1, wherein the generated 2D images for a given 4D Lightfield Data object are ordered based on the at least one geometrical parameter, and the matching operation between the search query image and the generated 2D images are performed based on the order.

### Claim 5 (depends on 4)

The method of claim 4, wherein the geometrical parameter is focal length and the matching operation between the search query image and the generated 2D images are performed based on a 1D ordering according to focal length.

### Claim 6 (depends on 5)

The method of claim 5, wherein the matching operation comprises an iteration on a subset of the plurality of generated 2D images, the iteration being based on feature distances between the search query image and 2D images selected from the plurality of generated 2D images according to focal length.

### Claim 7 (depends on 6)

The method of claim 6, wherein the matching operation comprises:
determining a first feature distance between the search query image and a first generated 2D image selected from the plurality of generated 2D images having a first index in the 1D ordering of the plurality of generated 2D images,
determining a second feature distance between the search query image and a second generated 2D image selected from the plurality of generated 2D images having a second index in the 1D ordering of the plurality of generated 2D images,
comparing the first feature distance to the second feature distance; and
selecting a further first generated 2D image and a further second generated 2D image based on the comparison result, the further first generated 2D image and the further second generated 2D image to be compared in a further search iteration.

### Claim 8 (depends on 4)

The method of claim 4, wherein the geometrical parameter is viewing direction and the matching operation between the search query image and the generated 2D images are performed based on a 2D ordering according to a grid of viewing directions.

### Claim 9

An apparatus for querying a database of 4D Lightfield Data objects, comprising a processing device configured to:
determine to query the database using a search query image;
generate, for each 4D Lightfield Data object in the database, multiple 2D images which differ from one another in at least one geometrical parameter;
perform a matching operation between the search query image and the generated 2D images to determine a target image, the target image being the generated 2D image which most closely matches the search query image according to a metric;
determine a target 4D Lightfield Data object, as the 4D Lightfield Data object from which the target image was generated; and
retrieve the target 4D Lightfield Data object in response to the query.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the search query image is determined based on user input.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein the search query image is a representative image generated from a 4D Lightfield Data query object.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein the generated 2D images for a given 4D Lightfield Data object are ordered based on the at least one geometrical parameter, and the matching operation between the search query image and the generated 2D images are performed based on the order.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the geometrical parameter is focal length and the matching operation between the search query image and the generated 2D images are performed based on a 1D ordering according to focal length.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the matching operation comprises an iteration on a subset of the plurality of generated 2D images, the iteration being based on feature distances between the search query image and 2D images selected from the plurality of generated 2D images according to focal length.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the matching operation comprises:
determining a first feature distance between the search query image and a first generated 2D image selected from the plurality of generated 2D images having a first index in the 1D ordering of the plurality of generated 2D images,
determining a second feature distance between the search query image and a second generated 2D image selected from the plurality of generated 2D images having a second index in the 1D ordering of the plurality of generated 2D images,
comparing the first feature distance to the second feature distance; and
selecting a further first generated 2D image and a further second generated 2D image based on the comparison result, the further first generated 2D image and the further second generated 2D image to be compared in a further search iteration.

### Claim 16 (depends on 12)

The apparatus of claim 12, wherein the geometrical parameter is viewing direction and the matching operation between the search query image and the generated 2D images are performed based on a 2D ordering according to a grid of viewing directions.

### Claim 17

A non-transitory computer readable storage medium having stored therein instructions for querying a database of 4D Lightfield Data objects, which, when executed by a computer, cause the computer to:
determine to query the database using a search query image;
generate, for each 4D Lightfield Data object in the database, multiple 2D images which differ from one another in at least one geometrical parameter;
perform a matching operation between the search query image and the generated 2D images to determine a target image, the target image being the generated 2D image which most closely matches the search query image according to a metric;
determine a target 4D Lightfield Data object, as the 4D Lightfield Data object from which the target image was generated; and
retrieve the target 4D Lightfield Data object in response to the query.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17 wherein the generated 2D images for a given 4D Lightfield Data object are ordered based on the at least one geometrical parameter, and the matching operation between the search query image and the generated 2D images are performed based on the order.

### Claim 19 (depends on 18)

The non-transitory computer readable storage medium of claim 18, wherein the geometrical parameter is focal length and the matching operation between the search query image and the generated 2D images are performed based on a 1D ordering according to focal length.

### Claim 20 (depends on 19)

The non-transitory computer readable storage medium of claim 19, wherein the matching operation comprises an iteration on a subset of the plurality of generated 2D images, the iteration being based on feature distances between the search query image and 2D images selected from the plurality of generated 2D images according to focal length.
