# US 8,279,500 B2 · System and method for integrated pair-wise registration of images using image based information and sensor coordinate and error information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,500 B2
- Document ID: US8279500B2
- Status: Granted
- Kind: B2
- Application number: 13/015,355
- Filed: 2011-01-27
- Published: 2012-08-02
- Granted: 2012-10-02
- Earliest claimed: 2011-01-27
- Assignees: Seiko Epson Corporation
- Inventors: Guoyi Fu; Mikhail Brusnitsyn
- Web page: https://patentgrep.com/patent/US8279500B2

## CPC

- H04N 1/047: Detection, control or error compensation of scanning velocity or position ({H04N1/0402 and } H04N1/17 take precedence)
- H04N 1/387: Composing, repositioning or otherwise {geometrically} modifying originals
- H04N 1/107: with manual scanning
- H04N 2201/0414: Scanning an image in a series of overlapping zones
- H04N 2201/0471: using dedicated detectors
- H04N 2201/04732: Detecting at infrequent intervals, e.g. once or twice per line for main-scan control
- H04N 2201/04787: by changing or controlling the addresses or values of pixels, e.g. in an array, in a memory, by interpolation
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Two images are stitched together through minimization of a cost function that consists of registration errors from image data of the two images, as well as the estimated errors from a set of sensors. The weight function in the cost function is derived from the confidence value of sensor estimation that considers the sensor errors including lift and off page as well as a measure of accuracy of the sensor readings. Weights are used to adjust image registration accuracy against sensor accuracy to produce a set of registration parameters that would best stitch the two images together. In order to handle large errors for initial registration parameters and to avoid local minima in the minimization process, the image pair may be registered in a lower resolution and then refined in a higher resolution.

## Claims

### Claim 1

A method for registering two image frames captured by a scanning device, the scanning device comprising an image capturing unit and a plurality of position sensors that supply position data indicating the relative position of the image capturing device when the image capturing device captures each of the two image frames, comprising:
reading position data from each of the plurality of position sensors when a first image frame i is captured and when a second image frame i+1 is captured;
estimating transformation parameters from the sensor position data that relate an image center of image frame i+1 to image frame i;
extracting an overlapping area between image frame i+1 and image frame i using the estimated transformation parameters from the sensor position data; and
refining the transformation parameters by minimizing a cost function integrating a registration error ES from sensor position data and a registration error EI from image data of the overlapping area to register image frame i+1 with image frame i.

### Claim 2 (depends on 1)

A method as in claim 1 wherein integrating the registration error ES and the registration error EI comprises weighting at least one of the registration error ES or the registration error EI.

### Claim 3 (depends on 2)

A method as in claim 2 wherein the weighting is a function of confidence in the sensor position data.

### Claim 4 (depends on 3)

A method as in claim 3 wherein the confidence is a measure of a semi-axis of error ellipse.

### Claim 5 (depends on 1)

A method as in claim 1 wherein the steps of reading, estimating, extracting, and refining are performed first in a first resolution and then performed again in a second resolution that is higher than the first resolution.

### Claim 6 (depends on 5)

A method as in claim 5 wherein the first resolution is selected based upon confidence in the sensor position data.

### Claim 7

A scanning device comprising:
an image capturing unit;
a plurality of position sensors that supply position data indicating the relative position of the image capturing device when the image capturing device captures one of a plurality of image frames; and
a processor that:
reads position data from each of the plurality of position sensors when a first image frame i is captured and when a second image frame i+1 is captured;
estimates transformation parameters from the sensor position data that relate an image center of image frame i+1 to image frame i; and
extracts an overlapping area between image frame i+1 and image frame i using the estimated transformation parameters from the sensor position data; and
refines the transformation parameters by minimizing a cost function integrating a registration error ES from sensor position data and a registration error EI from image data of the overlapping area to register image frame i+1 with image frame i.

### Claim 8 (depends on 7)

A scanning device as in claim 7 wherein integrating the registration error ES and the registration error EI comprises weighting at least one of the registration error ES or the registration error EI.

### Claim 9 (depends on 8)

A scanning device as in claim 8 wherein the weighting is a function of confidence in the sensor position data.

### Claim 10 (depends on 9)

A scanning device as in claim 9 wherein the confidence is a measure of a semi-axis of error ellipse.

### Claim 11 (depends on 7)

A scanning device as in claim 7 wherein the steps of reading, estimating, extracting, and refining are performed first in a first resolution and then performed again in a second resolution that is higher than the first resolution.

### Claim 12 (depends on 11)

A scanning device as in claim 11 wherein the first resolution is selected based upon confidence in the sensor position data.

### Claim 13

A non-transitory computer-readable medium encoded with computer-executable instructions for performing the following:
read position data from each of a plurality of position sensors when a first image frame i is captured and when a second image frame i+1 is captured;
estimates transformation parameters from the sensor position data that relate an image center of image frame i+1 to image frame i;
extract an overlapping area between image frame i+1 and image frame i using the estimated transformation parameters from the sensor position data; and
refine the transformation parameters by minimizing a cost function integrating a registration error ES from sensor position data and a registration error EI from image data of the overlapping area to register image frame i+1 with image frame i.

### Claim 14 (depends on 13)

The non-transitory computer-readable medium as in claim 13 wherein integrating the registration error ES and the registration error EI comprises weighting at least one of the registration error ES or the registration error EI.

### Claim 15 (depends on 14)

The non-transitory computer-readable medium as in claim 14 wherein the weighting is a function of confidence in the sensor position data.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium as in claim 15 wherein the confidence is a measure of a semi-axis of error ellipse.

### Claim 17 (depends on 13)

The non-transitory computer-readable medium as in claim 13 wherein the steps of reading, estimating, extracting, and refining are performed first in a first resolution and then performed again in a second resolution that is higher than the first resolution.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium as in claim 17 wherein the first resolution is selected based upon confidence in the sensor position data.
