# US 2019/0378255 A1 · Image processing apparatus, image processing method, and non-transitory computer-readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0378255 A1 - Document ID: US20190378255A1 - Status: Published application - Kind: A1 - Application number: 16/415,318 - Filed: 2019-05-17 - Published: 2019-12-12 - Earliest claimed: 2018-06-08 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Kaori Taya - Web page: https://patentgrep.com/patent/US20190378255A1 ## CPC - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - H04N 5/2625: {for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect (sequence generated by event triggered capturing H04N7/188)} - H04N 5/265: Mixing - H04N 23/76: by influencing the image signals - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - G06T 2207/20216: Image averaging - G06T 2207/20221: Image fusion; Image merging - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A single composite image is generated by selecting a pixel value for each pixel in any one of a plurality of captured images having been consecutively captured. A motion vector between at least two captured images among the plurality of captured images is calculated. Correction processing is performed, based on a motion vector corresponding to a pixel of interest in the composite image, on the pixel of interest or a pixel near the pixel of interest. The correction processing is performed by updating, using a bright pixel value, a pixel value of at least one pixel in a group of pixels associated with a path from the pixel of interest to a pixel indicated by the motion vector corresponding to the pixel of interest. ## Claims ### Claim 1 An image processing apparatus, comprising: a generating unit configured to generate a single composite image by selecting a pixel value for each pixel in any one of a plurality of captured images having been consecutively captured; a calculating unit configured to calculate a motion vector between at least two captured images among the plurality of captured images; and a correcting unit configured to perform correction processing, based on a motion vector corresponding to a pixel of interest in the composite image, on the pixel of interest or a pixel near the pixel of interest, wherein the correcting unit performs the correction processing by updating, using a bright pixel value, a pixel value of at least one pixel in a group of pixels associated with a path from the pixel of interest to a pixel indicated by the motion vector corresponding to the pixel of interest. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit performs the correction processing on any one of the pixels included in the group of pixels associated with the path. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit updates a pixel value of a correction target pixel to a brighter pixel value. ### Claim 4 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit updates, using a pixel value of any one of the pixels included in the group of pixels associated with the path, a pixel value of the correction processing target pixel. ### Claim 5 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit updates, using a brightest pixel value among the group of pixels associated with the path, a pixel value of the correction processing target pixel. ### Claim 6 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit refers to a motion vector calculated by the calculating unit with respect to a pixel selected by the generating unit as a pixel of interest in the composite image, in order to correct a pixel value of the pixel of interest. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit specifies, as a group of pixels associated with the path, pixels through which a line segment passes, the line segment connecting the pixel of interest and a pixel separate from the pixel of interest in a direction of a motion vector corresponding to the pixel of interest. ### Claim 8 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit corrects, based on a pixel value of a pixel in the composite image and a pixel value of a pixel separate from the pixel in a direction of a motion vector corresponding to the pixel, the pixel value of the separate pixel. ### Claim 9 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit weights a pixel value of a pixel separate from the pixel of interest in a direction of a motion vector corresponding to the pixel of interest, in accordance with a distance between the separate pixel and a line segment connecting the pixel of interest and the separate pixel, and then the correcting unit corrects the pixel value of the pixel of interest based on the weighted pixel value. ### Claim 10 (depends on 1) The image processing apparatus according to claim 1, wherein the generating unit selects a pixel having a maximum luminance value among pixels at a pixel of interest position of the plurality of captured images, and the generating unit selects the selected pixel as a pixel at the pixel of interest position in the composite image. ### Claim 11 (depends on 1) The image processing apparatus according to claim 1, wherein the generating unit selects a pixel having a minimum luminance value among the pixels at a pixel of interest position of the plurality of captured images, and the generating unit selects the selected pixel as a pixel at the pixel of interest position in the composite image. ### Claim 12 (depends on 1) The image processing apparatus according to claim 1, wherein the calculating unit calculates a motion vector between the captured image and a captured image having been captured immediately before the captured image. ### Claim 13 (depends on 1) The image processing apparatus according to claim 1, wherein the calculating unit calculates a motion vector between the captured image and a captured image having been captured immediately after the captured image. ### Claim 14 (depends on 1) The image processing apparatus according to claim 1, wherein the plurality of captured images is a plurality of frames included in continuous shots or a moving picture, and the calculating unit corrects a motion vector between the frames based on an exposure time and a frame rate of the captured image. ### Claim 15 (depends on 1) The image processing apparatus according to claim 1, wherein the generating unit generates a single composite image using a pixel selected from a plurality of captured images that inverse γ conversion has been performed, the calculating unit calculates a motion vector between at least two captured images among the plurality of captured images that the inverse γ conversion has been performed, and the correcting unit corrects the composite image based on a motion vector calculated by the calculating unit with respect to a pixel selected by the generating unit as a pixel in the composite image, and the correcting unit performs γ conversion on a result obtained by α-blending the corrected composite image and an average image of the plurality of captured images that the inverse γ conversion has been performed. ### Claim 16 (depends on 1) The image processing apparatus according to claim 1, wherein in a case where bright pixels are included in the group of pixels associated with the path, the correcting unit performs the correction processing in such a manner as to cause the bright pixels to be continued. ### Claim 17 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit specifies a pixel whose pixel value is to be updated by the correction processing, using a motion vector of the pixel of interest. ### Claim 18 (depends on 1) The image processing apparatus according to claim 1, wherein the correcting unit determines, using the motion vector, a value from which a pixel value of the correction processing target pixel is updated. ### Claim 19 An image processing method performed by an image processing apparatus, the method comprising: generating a single composite image by selecting a pixel value for each pixel in any one of a plurality of captured images having been consecutively captured; calculating a motion vector between at least two captured images among the plurality of captured images; and performing correction processing, based on a motion vector corresponding to a pixel of interest in the composite image, on the pixel of interest or a pixel near the pixel of interest, wherein the correction processing is performed by updating, using a bright pixel value, a pixel value of at least one pixel in a group of pixels associated with a path from the pixel of interest to a pixel indicated by the motion vector corresponding to the pixel of interest. ### Claim 20 A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as: a generating unit configured to generate a single composite image by selecting a pixel value for each pixel in any one of a plurality of captured images having been consecutively captured; a calculating unit configured to calculate a motion vector between at least two captured images among the plurality of captured images; and a correcting unit configured to perform correction processing, based on a motion vector corresponding to a pixel of interest in the composite image, on the pixel of interest or a pixel near the pixel of interest, wherein the correcting unit performs the correction processing by updating, using a bright pixel value, a pixel value of at least one pixel in a group of pixels associated with a path from the pixel of interest to a pixel indicated by the motion vector corresponding to the pixel of interest.