# US 10,872,396 B2 · Image stitching method and device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,396 B2 - Document ID: US10872396B2 - Status: Granted - Kind: B2 - Application number: 16/094,640 - Filed: 2017-04-24 - Published: 2019-04-11 - Granted: 2020-12-22 - Earliest claimed: 2016-04-22 - Assignees: Cyclomedia Technology B.V. - Applicants: Cyclomedia Technology B.V. - Inventors: Christian Leonard Lucien Bartels; Bart Johannes Beers - Web page: https://patentgrep.com/patent/US10872396B2 ## CPC - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration - G06T 2207/20221: Image fusion; Image merging - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present invention is related to an image stitching method. It is further related to an image stitching device and to a computer readable medium carrying instructions for performing such method. The method according to the invention includes determining a pixel value of a pixel at an interpolation position in the region to be stitched using pixel values of corresponding pixels in a plurality of sub-images. According to the invention, the interpolation position corresponds to a weighted sum of positions of the corresponding pixels. The weighting factor for the position of a corresponding pixel in a given sub-image depends on a relative distance of at said interpolation position to a border of the region to be stitched associated with that sub-image. ## Claims ### Claim 1 A method for stitching a plurality of globally aligned sub-images in a region to be stitched, the method comprising: determining a pixel value of a pixel at an interpolation position in the region to be stitched using pixel values of corresponding pixels in said plurality of sub-images; wherein the interpolation position corresponds to a weighted sum of positions of the corresponding pixels; wherein the weighting factor for the position of a corresponding pixel in a given sub-image depends on a relative distance of said interpolation position to a border of the region to be stitched associated with that sub-image. ### Claim 2 (depends on 1) The method according to claim 1, wherein said relative distance corresponds to a distance of said interpolation position to a border of the region to be stitched associated with that sub-image relative to the sum of those distances for all sub-images. ### Claim 3 (depends on 2) The method according to claim 2, wherein the relative distance is given by: d i ∑ n = 1 N ⁢ d n wherein di is the distance of said interpolation position to a border of the region to be stitched associated with sub-image i, and wherein N is the total number of sub-images. ### Claim 4 (depends on 1) The method according to claim 1, comprising: finding a set of corresponding pixels in the plurality of globally aligned sub-images; determining an interpolation position that corresponds to the weighted sum of the positions of the set of corresponding pixels. ### Claim 5 (depends on 4) The method according to claim 4, wherein said determining comprises: a) guessing an interpolation position; b) determining the weighting factors for the corresponding pixels; c) calculating an estimated interpolation position by applying the weighted sum of the positions of the corresponding pixels using the determined weighting factors; d) determining an error between the estimated interpolation position and the guessed interpolation position and repeating steps a)-d) if the error exceeds a predefined threshold. ### Claim 6 (depends on 1) The method according to claim 1, comprising: determining an interpolation position in the region to be stitched; finding the corresponding pixels of which the pixel values are to be used for determining the pixel value of the pixel at the determined interpolation position. ### Claim 7 (depends on 6) The method according to claim 6, wherein said finding the corresponding pixels comprises finding the corresponding pixels such that an estimated interpolation position, which is computed using a weighted sum of the positions of those corresponding pixels, substantially corresponds to the determined interpolation position, wherein the weighting factors to be used for said computation are calculated using the determined interpolation position. ### Claim 8 (depends on 6) The method according to claim 6, wherein said finding the corresponding pixels comprises: a) guessing weighted test vectors for finding corresponding pixels such that when an interpolation position would be calculated using those pixels, it would at least substantially correspond to the determined interpolation position, said weighted test vectors being calculated using the determined interpolation position; b) determining an error between pixels found using the weighted test vectors and repeating steps a)-b) if the error exceeds a predefined threshold. ### Claim 9 (depends on 6) The method according to claim 6, wherein said finding the corresponding pixels comprises: a) guessing a set of positions of corresponding pixels; b) determining the weighting factors for the corresponding pixels; c) calculating an estimated interpolation position by applying the weighted sum of the guessed positions of the corresponding pixels; d) determining an error between the estimated interpolation position and the determined interpolation position and repeating steps a)-d) if the error exceeds a predefined threshold. ### Claim 10 (depends on 1) The method according to claim 1, wherein the weighting factor for the position of a corresponding pixel in a given sub-image equals the relative distance associated with that sub-image. ### Claim 11 (depends on 1) The method according to claim 1, wherein the weighting factor decreases if the distance of said pixel or said interpolation position to said border decreases. ### Claim 12 (depends on 1) The method according to claim 1, wherein said distance is the shortest distance to the border. ### Claim 13 (depends on 1) The method according to claim 1, wherein the corresponding pixels in the plurality of sub-images are pixels that relate to the same region or point of an object or feature that is imaged in said plurality of sub-images. ### Claim 14 (depends on 13) The method according to claim 13, further comprising using a motion estimation technique to find corresponding pixels in said plurality of sub-images, or a technique to find dense correspondences. ### Claim 15 (depends on 1) The method according to claim 1, wherein the border of the region to be stitched associated with a sub-image corresponds to an edge of that sub-image in the aligned plurality of sub-images. ### Claim 16 (depends on 15) The method according to claim 15, wherein the region to be stitched corresponds to an entire region in which at least two or more sub-images are overlapping. ### Claim 17 (depends on 1) The method according to claim 1, wherein the region to be stitched is a region centred about a curve through a region in said plurality of images in which at least two of said sub-images are overlapping. ### Claim 18 (depends on 17) The method according to claim 17, further comprising: determining a correction benchmark using the differences in pixel values between the respective corresponding pixels for a plurality of pixels in the region to be stitched; comparing the correction benchmark to a predefined threshold; if the correction benchmark exceeds the predefined threshold, extending the region to be stitched. ### Claim 19 (depends on 17) The method according to claim 17, further comprising: determining a displacement vector field in the region to be stitched that indicates a difference in position between corresponding pixels in different sub-images; calculating the curve using the displacement vector field. ### Claim 20 (depends on 19) The method according to claim 19, wherein the curve is calculated using a least cost optimization technique. ### Claim 21 (depends on 20) The method according to claim 20, wherein the curve is configured to avoid regions in which the displacement vectors are relatively large. ### Claim 22 (depends on 1) The method according to claim 1, further comprising determining a pixel value of the pixel at the interpolation position in the region to be stitched by weighting the pixel values of the corresponding pixels associated with that interpolation position using further weighting factors. ### Claim 23 (depends on 22) The method according to claim 22, wherein the further weighting factors are identical to the weighting factors. ### Claim 24 (depends on 1) A device for stitching a plurality of globally aligned sub-images in a region to be stitched, comprising: a memory for holding said plurality of sub-images; a stitching region determining unit to determine the region to be stitched; and a processor configured to construct a final image using the plurality of sub-images, said constructing comprising performing the method as defined in claim 1 to stitch the sub-images in the region to be stitched. ### Claim 25 (depends on 1) A non-transitory computer readable storage medium comprising instructions that, when run on a processor, instruct this processor to perform the method as described in claim 1.