# US 12,119,025 B2 · Image processing device, image processing method, and image processing program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,119,025 B2 - Document ID: US12119025B2 - Status: Granted - Kind: B2 - Application number: 17/599,065 - Filed: 2020-04-03 - Published: 2022-06-23 - Granted: 2024-10-15 - Earliest claimed: 2019-04-18 - Assignees: Nippon Telegraph and Telephone Corporation - Applicants: Nippon Telegraph and Telephone Corporation - Inventors: Masato Ono; Takahide Hoshide; Yasushi Matsubara; Shinji Fukatsu; Kenichi Minami - Web page: https://patentgrep.com/patent/US12119025B2 ## CPC - G06T 5/50: using two or more images, e.g. averaging or subtraction - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/16: {using multiple overlapping images; Image stitching} - G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - H04N 1/387: Composing, repositioning or otherwise {geometrically} modifying originals - H04N 5/66: Transforming electric information into light information (scanning details H04N3/00) - H04N 23/60: Control of cameras or camera modules - G06T 2207/10016: Video; Image sequence - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30196: Human being; Person - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract To suppress an unnatural movement of a moving object at a connection point when a plurality of pieces of video data are connected to each other. A frame acquisition unit 22 acquires pieces of frame data captured at the same time, from the pieces of video data input from cameras C1 and C2. A prohibited-region setting unit 24 sets a prohibited region that is not set as a calculation target when a seam is calculated, based on a position of an object detected in an overlap region between adjacent pieces of frame data and a movement direction of the object. A seam calculation unit 25 calculates a seam between pieces of frame data adjacent to each other, without setting a pixel included in the prohibited region as the calculation target of seam calculation. A connection-frame output unit 26 connects pieces of frame data in accordance with the seam and outputs connection frame data 15. ## Claims ### Claim 1 An image processing device that stitches a plurality of pieces of image data obtained by capturing portions of an image capturing region with an overlap, the image processing device comprising: a frame acquisition unit, including one or more processors, configured to acquire a plurality of pieces of frame data captured at the same time, from the plurality of pieces of image data; an object detection unit, including one or more processors, configured to detect a target object in an overlap region in which pieces of the frame data adjacent to each other overlap each other; a prohibited-region setting unit, including one or more processors, configured to set, in the overlap region, a prohibited region through which a seam does not pass; a seam calculation unit, including one or more processors, configured to calculate the seam not to pass through the prohibited region; and a connection-frame output unit, including one or more processors, configured to connect the plurality of pieces of frame data to each other in accordance with the seam and output connection frame data obtained by the connection, wherein the prohibited-region setting unit is configured to: (i) determine, in the overlap region, a first region that encompasses the detected target object, (ii) determine a second region which spans from a side of the first region to a side of the overlap region on an entrance direction side of the target object, and (iii) set the prohibited region as including both of the first region and the second region. ### Claim 2 (depends on 1) The image processing device according to claim 1, wherein the prohibited-region setting unit is configured to set, as the prohibited region, a region from the position of the target object to a side of the overlap region on a traveling direction side of the target object after detected positions of the target object on the frame data in the overlap region are substantially the same. ### Claim 3 An image processing method by an image processing device for stitching a plurality of pieces of image data obtained by capturing portions of an image capturing region with an overlap, the image processing method comprising: acquiring a plurality of pieces of frame data captured at the same time, from the plurality of pieces of image data; detecting a target object in an overlap region in which pieces of the frame data adjacent to each other overlap each other; setting, in the overlap region, a prohibited region through which a seam does not pass; and outputting connection frame data by connecting the plurality of pieces of frame data to each other in accordance with the seam, wherein setting the prohibited region comprises: (i) determining, in the overlap region, a first region that encompasses the detected target object, (ii) determining a second region which spans from a side of the first region to a side of the overlap region on an entrance direction side of the target object, and (iii) setting the prohibited region as including both of the first region and the second region. ### Claim 4 (depends on 3) The image processing method according to claim 3, wherein, in setting the prohibited region, a region from the position of the target object to a side of the overlap region on a traveling direction side of the target object is set as the prohibited region after detected positions of the target object on the frame data in the overlap region are substantially the same. ### Claim 5 A non-transitory computer readable storage medium having stored thereon an image processing program for causing one or more processors of an image processing device to perform an image processing method for stitching a plurality of pieces of image data obtained by capturing portions of an image capturing region with an overlap, the image processing method comprising: acquiring a plurality of pieces of frame data captured at the same time, from the plurality of pieces of image data; detecting a target object in an overlap region in which pieces of the frame data adjacent to each other overlap each other; setting, in the overlap region, a prohibited region through which a seam does not pass; and outputting connection frame data by connecting the plurality of pieces of frame data to each other in accordance with the seam, wherein setting the prohibited region comprises: (i) determining, in the overlap region, a first region that encompasses the detected target object, (ii) determining a second region which spans from a side of the first region to a side of the overlap region on an entrance direction side of the target object, and (iii) setting the prohibited region as including both of the first region and the second region. ### Claim 6 (depends on 5) The non-transitory computer readable storage medium according to claim 5, wherein, in setting the prohibited region, a region from the position of the target object to a side of the overlap region on a traveling direction side of the target object is set as the prohibited region after detected positions of the target object on the frame data in the overlap region are substantially the same.