# US 2015/0147000 A1 · Method and apparatus for fusion of images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0147000 A1
- Document ID: US20150147000A1
- Status: Published application
- Kind: A1
- Application number: 14/408,161
- Filed: 2013-05-30
- Published: 2015-05-28
- Earliest claimed: 2012-06-15
- Assignees: Thomson Licensing
- Applicants: Thomson Licensing
- Inventors: Jordi Salvador Marcos; Malte Borsum; Axel Kochale
- Web page: https://patentgrep.com/patent/US20150147000A1

## CPC

- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 15/10: Geometric effects
- H04N 5/2624: {for obtaining an image which is composed of whole input images, e.g. splitscreen}
- H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation}
- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and an apparatus for improving a main image by fusing the richer information contained in a secondary image are described. A 3D structure of objects contained in the secondary image is retrieved and a parallax-corrected version of the secondary image is generated using the 3D structure. For this purpose a camera pose for which a projection of the 3D structure of the objects contained in the secondary image best resembles the perspective in the main image is determined and the parallax-corrected version of the secondary image is synthesized based on the determined camera pose. The parallax-corrected version of the secondary image is then fused with the main image.

## Claims

### Claim at position 1

1-7. (canceled)

### Claim 8

A method for fusing a secondary image with a main image, the method comprising:
retrieving a 3D structure of objects contained in the secondary image;
generating a parallax-corrected version of the secondary image using the 3D structure by determining a camera pose for which a projection of the 3D structure of the objects contained in the secondary image best resembles the perspective in the main image and synthesizing the parallax-corrected version of the secondary image based on the determined camera pose; and
fusing the parallax-corrected version of the secondary image with the main image.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the 3D structure of the secondary image is retrieved from depth information or a light-field description provided for the secondary image.

### Claim 10 (depends on 8)

The method according to claim 8, wherein the parallax-corrected version of the secondary image is synthesized by performing a photo-realistic warping of the secondary image.

### Claim 11 (depends on 8)

The method according to claim 8, further comprising interpolating a boundary region between the main image and the fused parallax-corrected version of the secondary image.

### Claim 12 (depends on 8)

The method according to claim 8, wherein the main image is a panoramic image and the secondary image is a more detailed image of an object in the panoramic image.

### Claim 13 (depends on 8)

The method according to claim 8, wherein the main image and the secondary image are two images of a set of images.

### Claim 14

An apparatus for fusing a secondary image with a main image, the apparatus comprising:
a 3D structure retrieval stage configured to retrieve a 3D structure of objects contained in the secondary image;
a parallax correction stage configured to generate a parallax-corrected version of the secondary image using the 3D structure by determining a camera pose for which a projection of the 3D structure of the objects contained in the secondary image best resembles the perspective in the main image and to synthesize the parallax-corrected version of the secondary image based on the determined camera pose; and
an image fusion stage configured to fuse the parallax-corrected version of the secondary image with the main image.

### Claim 15 (depends on 14)

The apparatus according to claim 14, wherein the 3D structure retrieval stage is configured to retrieve the 3D structure of the secondary image from depth information or a light-field description provided for the secondary image.

### Claim 16 (depends on 14)

The apparatus according to claim 14, wherein the parallax correction stage is configured to synthesize the parallax-corrected version of the secondary image by performing a photo-realistic warping of the secondary image.

### Claim 17 (depends on 14)

The apparatus according to claim 14, wherein the image fusion stage is configured to interpolate a boundary region between the main image and the fused parallax-corrected version of the secondary image.

### Claim 18 (depends on 14)

The apparatus according to claim 14, wherein the main image is a panoramic image and the secondary image is a more detailed image of an object in the panoramic image.

### Claim 19 (depends on 14)

The apparatus according to claim 14, wherein the main image and the secondary image are two images of a set of images.
