# US 10,481,482 B2 · Method and device for generating panoramic images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,482 B2
- Document ID: US10481482B2
- Status: Granted
- Kind: B2
- Application number: 15/873,018
- Filed: 2018-01-17
- Published: 2019-01-10
- Granted: 2019-11-19
- Earliest claimed: 2017-07-05
- Assignees: Shanghai Xiaoyi Technology Co., Ltd.
- Applicants: Shanghai Xiaoyi Technology Co., Ltd.
- Inventors: Junzhou Mu
- Web page: https://patentgrep.com/patent/US10481482B2

## CPC

- G03B 37/04: with cameras or projectors providing touching or overlapping fields of view
- G03B 35/02: by sequential recording
- G03B 37/02: with scanning movement of lens or cameras
- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- H04N 7/181: {for receiving images from a plurality of remote sources}
- H04N 13/106: Processing image signals (for multi-view video sequence encoding H04N19/597)
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- H04N 13/296: Synchronisation thereof; Control thereof
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- H04N 2013/0088: {Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image}
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and a device for generating panoramic images. The method includes capturing images from positions in a circular array, projecting the captured images into corresponding equirectangular images, extracting a set of intermediate images from the equirectangular images, and blending the set of intermediate images to generate panoramic images.

## Claims

### Claim 1

A method for generating panoramic images, comprising:
capturing images by at least one camera from a plurality of positions in a circular array;
projecting, by a processor, the captured images into corresponding equirectangular images;
extracting, by the processor, a set of intermediate images from the equirectangular images; and
blending, by the processor, the set of intermediate images to generate panoramic images,
wherein the panoramic images comprise a first panoramic image including scenes for presentation to a right eye of a user and a second panoramic image including scenes for presentation to a left eye of the user.

### Claim 2 (depends on 1)

The method of claim 1, wherein projecting the captured images further comprises:
determining, by the processor, coordinates of pixels in a first coordinate system corresponding to the captured images;
looking up the coordinates in a mapping table;
obtaining corresponding coordinates of pixels in the equirectangular images from the mapping table; and
supplying values of the pixels in the captured images to pixels in the corresponding coordinates of the equirectangular images.

### Claim 3 (depends on 1)

The method of claim 1, wherein extracting a set of intermediate images comprises:
extracting a center portion of each of the equirectangular images as an intermediate image, to obtain the set of intermediate images.

### Claim 4 (depends on 1)

The method of claim 1, wherein extracting a set of intermediate images comprises:
determining from the projected equirectangular images:
a first equirectangular image corresponding to a first image captured at a first position;
a second equirectangular image corresponding to a second image captured at a second position; and
a third equirectangular image corresponding to a third image captured at a third position;
wherein the second position is on a left side of the first position, and the third position is on a right side of the first position in the circular array;
extracting, as a first intermediate image, a first portion of the first equirectangular image that overlaps with the second equirectangular image; and
extracting, as a second intermediate image, a second portion of the first equirectangular image that overlaps with the third equirectangular image.

### Claim 5 (depends on 4)

The method of claim 4, wherein:
the set of intermediate images comprises a first subset of intermediate images and a second subset of intermediate images; and
blending the set of intermediate images comprises:
determining a first overlapping area between two intermediate images in the first subset, and blending the two intermediate images in the first subset, based on the first overlapping area, to obtain the first panoramic image; and
determining a second overlapping area between two intermediate images in the second subset, and blending the two intermediate images in the second subset, based on the second overlapping area, to obtain the second panoramic image.

### Claim 6 (depends on 1)

The method of claim 1, wherein blending the set of the intermediate images comprises:
determining an overlapping area between two intermediate images in the set of the intermediate images; and
blending the two intermediate images based on the overlapping area.

### Claim 7 (depends on 1)

The method of claim 1, wherein capturing images from positions in a circular array comprises:
capturing the images by sixteen cameras respectively disposed at the positions, facing radially outward.

### Claim 8 (depends on 1)

The method of claim 1, wherein capturing images from positions in a circular array comprises:
capturing a first image by one camera at a first position; and
rotating the camera by an angle to a second position to capture a second image.

### Claim 9 (depends on 8)

The method of claim 8, wherein the angle is 22.5 degrees.

### Claim 10

A device for generating panoramic images, comprising:
a camera rig;
at least one camera, mounted on the camera rig and configured to capture images from a plurality of positions in a circular array;
a memory, configured to store the captured images and instructions; and
a processor, configured to execute the instructions to:
project the captured images into corresponding equirectangular images;
extract a set of intermediate images from the equirectangular images; and
blend the set of intermediate images to generate panoramic images,
wherein the panoramic images comprise a first panoramic image including scenes for presentation to a right eye of a user and a second panoramic image including scenes for presentation to a left eye of the user.

### Claim 11 (depends on 10)

The device of claim 10, wherein the processor further executes the instructions to project the captured images into corresponding equirectangular images by:
determining coordinates of pixels in the captured images in a first coordinate system;
looking up the coordinates in a mapping table;
obtaining corresponding coordinates of pixels in the equirectangular images in a second coordinate system from the mapping table; and
supplying values of the pixels in the captured images to pixels in the corresponding coordinates of the equirectangular images.

### Claim 12 (depends on 10)

The device of claim 10, wherein the processor further executes the instructions to extract a set of intermediate images by:
extracting a center portion of each of the equirectangular images as intermediate images, to obtain the set of intermediate images.

### Claim 13 (depends on 10)

The device of claim 10, wherein the processor further executes the instructions to extract a set of intermediate images by:
determining from the projected equirectangular images:
a first equirectangular image corresponding to a first image captured at a first position;
a second equirectangular image corresponding to a second image captured at a second position; and
a third equirectangular image corresponding to a third image captured at a third position;
wherein the second position is on a left side of the first position, and the third position is on a right side of the first position in the circular array;
extracting, as a first intermediate image, a first portion of the first equirectangular image that overlaps with the second equirectangular image; and
extracting, as a second intermediate image, a second portion of the first equirectangular image that overlaps with the third equirectangular image.

### Claim 14 (depends on 13)

The device of claim 13, wherein:
the set of intermediate images comprises a first subset of intermediate images and a second subset of intermediate images; and
the processor further executes the instructions to blend the set of intermediate images by:
determining a first overlapping area between two intermediate images in the first subset of intermediate images, and blending the two intermediate images based on the first overlapping area to obtain the first panoramic image; and
determining a second overlapping area between two intermediate images in the second subset of intermediate images, and blending the two intermediate images based on the second overlapping area to obtain the second panoramic image.

### Claim 15 (depends on 10)

The device of claim 10, wherein the processor further executes the instructions to blend the set of intermediate images by:
determining an overlapping area between two intermediate images in the set of intermediate images; and
blending the two intermediate images based on the overlapping area.

### Claim 16 (depends on 10)

The device of claim 10, wherein the device includes sixteen cameras respectively disposed at the positions, facing radially outward.

### Claim 17 (depends on 10)

The device of claim 10, further comprising a rotation unit, wherein the device includes one camera, and captures a first image by the camera at a first position, and rotates, via the rotation unit, the camera by an angle to a second position to capture a second image.

### Claim 18 (depends on 17)

The device of claim 17, wherein the angle is 22.5 degrees.