# US 10,440,267 B2 · System and method for image stitching

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,440,267 B2
- Document ID: US10440267B2
- Status: Granted
- Kind: B2
- Application number: 15/997,708
- Filed: 2018-06-05
- Published: 2018-10-04
- Granted: 2019-10-08
- Earliest claimed: 2016-05-31
- Assignees: Tower Spring Global Limited
- Applicants: Tower Spring Global Limited
- Inventors: Eric Kwong Hang Tsang; Yuntao Sun
- Web page: https://patentgrep.com/patent/US10440267B2

## CPC

- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- H04N 5/265: Mixing
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Digital logic circuitry includes a plurality of logic blocks and memories that execute a method to stitch images. The method includes capturing the images with a plurality of pixel sensors, providing blocks of intensity values captured by the pixel sensors to input/output (I/O) pins of the digital logic circuitry, processing the intensity values with the digital logic circuitry, and stitching the processed images into a stitched image.

## Claims

### Claim 1

A camera, comprising:
a lens;
a plurality of image sensors that capture panoramic images;
random access memory that connects to the image sensors and stores pixel values of the panoramic images; and
a digital logic circuit having input/output (I/O) pins connected to the random access memory and including logic blocks and memories that obtain blocks of the pixel values of the panoramic images from the random access memory, save a portion of each of the blocks to each of the memories, calculate output pixel values based on pre-selected pixels in the blocks, generate output images from the calculated output pixel values, and stitch the output images with blending data to produce a 360° image,
wherein the digital logic circuit calculates the output pixel values based on a cubic interpolating coefficient generator represented by:
pixel
⁡
(
ux
,
uy
)
=
∑
i
=
0
3
⁢
⁢
∑
j
=
0
3
⁢
⁢
wx
⁡
(
i
,
ux
)
⁢
wy
⁡
(
j
,
uy
)
⁢
d
16
⁡
(
i
,
j
)
where d16 are the pixel values captured by the image sensors, and wx(i)=f(i, ux) and wy(j)=f(j, uy) are interpolation coefficients generated by the cubic interpolating coefficient generator with the following cubic interpolation functions:
f
⁡
(
0
,
x
)
=
-
3
×
x
3
+
6
×
x
2
-
3
×
x
4
,
⁢
f
⁡
(
1
,
x
)
=
5
×
x
3
-
9
×
x
2
+
4
4
,
⁢
f
⁡
(
2
,
x
)
=
-
5
×
x
3
+
6
×
x
2
+
3
×
x
4
,
⁢
f
⁡
(
3
,
x
)
=
3
×
x
3
-
3
×
x
2
4
.

### Claim 2 (depends on 1)

The camera of claim 1, wherein the digital logic circuit corrects position distortion and color distortion of the panoramic images, wherein the position distortion includes position errors from the image sensors and the lens of the camera and radial distortion from the lens of the camera.

### Claim 3 (depends on 1)

The camera of claim 1, wherein the blending data are generated by:
pre-selecting pixels that are at an edge of the output images;
dividing the pre-selected pixels into a plurality of groups;
saving the pixel values of each of the groups to the memories of the logic blocks of the digital logic circuit;
applying a matrix to coordinates of the pixels in the groups; and
generating, with the logic blocks of the digital logic circuit and by performing logic calculations to each of the groups, the blending data,
wherein the logic calculations include subtraction, summation, and multiplication.

### Claim 4 (depends on 1)

The camera of claim 1, wherein each of the logic blocks of the digital logic circuit calculates the output pixel values for the pre-selected pixels in parallel with each other.

### Claim 5 (depends on 1)

The camera of claim 1, wherein a number of the blocks of the panoramic images is dependent upon a number of the logic blocks in the digital logic circuit.

### Claim 6 (depends on 1)

The camera of claim 1, wherein the digital logic circuit calculates, for each output pixel of the output images, coordinate points in the blocks of the panoramic images that correspond to the output pixel of the output image.