# US 10,484,601 B2 · Image processing apparatus, image processing method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,601 B2 - Document ID: US10484601B2 - Status: Granted - Kind: B2 - Application number: 15/249,058 - Filed: 2016-08-26 - Published: 2017-03-02 - Granted: 2019-11-19 - Earliest claimed: 2015-08-31 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Kyohei Kikuta - Web page: https://patentgrep.com/patent/US10484601B2 ## CPC - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - H04N 1/00: Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof - H04N 5/2624: {for obtaining an image which is composed of whole input images, e.g. splitscreen} - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image processing apparatus according to the present disclosure includes an acquisition unit configured to acquire a position for identifying a region in which a plurality of images are combined in an overlap region, the plurality of images being obtained in a partially-overlapping manner, a detection unit configured to detect an edge in vicinity of the position for identifying the region in the plurality of images, and a combining unit configured to combine the plurality of images to generate an output image by blending pixel values of pixels constituting the plurality of images, in vicinity of the position for identifying the region, wherein the combining unit sets a parameter for controlling the region in which the plurality of images are combined, according to a correlation between the position for identifying the region and the edge. ## Claims ### Claim 1 An image processing apparatus for generating a panoramic image by combining a plurality of images in a partially-overlapping manner, comprising: one or more processors; and one or more memory storing instructions executable by the one or more processors, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: acquire a position of a boundary line for combining a first image and a second image partially overlapping the first image in an overlap region, the first image and the second image being two of the plurality of images; detect an edge component of an object in vicinity of the position of the boundary line in either of the first image and second image; set a blend width of a region, in the overlap region and including the boundary line, where pixel values of pixels constituting the first image and pixel values of pixels constituting the second image are blended, wherein the blend width is set based on the position of the boundary line and the edge component; and combine the first image and the second image by blending the pixel values of the pixels constituting the first image and the pixel values of the pixels constituting the second image, in the region of the set blend width in the overlap region. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: in a case in which the position of the boundary line and the edge component match each other, set the blend width of the region in which the first image and the second image are combined, to be smaller than in a case in which the position of the boundary line and the edge component do not match each other. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: calculate, as a correlation, a matching degree between a position and a direction of the boundary line and a position and a direction of the edge component. ### Claim 4 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: calculate an evaluation value for evaluating a matching degree between the position of the boundary line and the edge component, and set the blend width according to the evaluation value. ### Claim 5 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: acquire a maximum blend width comprising an upper limit of the blend width, and set the blend width to the maximum blend width or smaller. ### Claim 6 (depends on 5) The image processing apparatus according to claim 5, wherein the maximum blend width is set based on an instruction issued by a user. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: detect, as the edge component, a difference in characteristic between two partial regions obtained by division in a direction of the boundary line. ### Claim 8 (depends on 7) The image processing apparatus according to claim 7, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: divide vicinity of the boundary line into blocks, and set, for each block, a blend width so as to narrow the blend width in a case in which a difference in characteristic between partial regions is large, and to widen the blend width in a case in which a difference in characteristic between partial regions is small. ### Claim 9 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: detect, as the edge component, an amount representing a degree of change in pixel value. ### Claim 10 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: detect a frequency characteristic as the edge component. ### Claim 11 (depends on 1) The image processing apparatus according to claim 1, wherein the image includes distance information representing a distance from a shooting viewpoint for each region represented by pixels, and wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: detect an edge in distance information related to a position and a direction of the boundary line in the first image and the second image. ### Claim 12 An image processing method for generating a panoramic image by combining a plurality of images in a partially-overlapping manner, comprising: acquiring a position of a boundary line for combining a first image and a second image partially overlapping the first image in an overlap region, the first image and the second image being two of the plurality of images; detecting an edge component of an object in vicinity of the position of the boundary line in either of the first image and second image; setting a blend width of a region, in the overlap region and including the boundary line, where pixel values of pixels constituting the first image and pixel values of pixels constituting the second image are blended, wherein the blend width is set based on the position of the boundary line and the edge component; and combining the first image and the second image by blending the pixel values of the pixels constituting the first image and the pixel values of the pixels constituting the second image, in the region of the set blend width in the overlap region. ### Claim 13 A non-transitory computer-readable storage medium storing a program for causing a computer to execute an image processing method for generating a panoramic image by combining a plurality of images in a partially-overlapping manner, the image processing method comprising: acquiring a position of a boundary line for combining a first image and a second image partially overlapping the first image in an overlap region, the first image and the second image being two of the plurality of images; detecting an edge component of an object in vicinity of the position of the boundary line in either of the first image and second image; setting a blend width of a region, in the overlap region and including the boundary line, where pixel values of pixels constituting the first image and pixel values of pixels constituting the second image are blended, wherein the blend width is set based on the position of the boundary line and the edge component; and combining the first image and the second image by blending the pixel values of the pixels constituting the first image and the pixel values of the pixels constituting the second image, in the region of the set blend width in the overlap region. ### Claim 14 (depends on 1) The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to: acquire a boundary image indicating whether each pixel is a pixel positioned on the boundary line, generate pre-blend image, which is the same size as the boundary image, by selecting any of the pixels in the first image and the second image, and generate an edge image indicating whether each pixel is an edge by detecting an edge in the pre-blend image as the feature amount, and determine whether a block including the boundary line in the boundary image matches a block including the edge in the edge image, and set the blend width according to the result of determining. ### Claim 15 (depends on 1) The image processing apparatus according to claim 1, wherein the boundary line is a line in a direction orthogonal to a direction in which the first image and the second image about each other, and the blend width is a length in a direction in which the first image and the second image about each other. ### Claim 16 (depends on 1) The image processing apparatus according to claim 1, wherein the number of pixel pairs the pixel values of which are to be blended by the combining differs depending on the set blend width.