# US 2018/0025533 A1 · Image processing apparatus for determining image clip position, method thereof, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025533 A1 - Document ID: US20180025533A1 - Status: Published application - Kind: A1 - Application number: 15/648,294 - Filed: 2017-07-12 - Published: 2018-01-25 - Earliest claimed: 2016-07-22 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Masanori Sato; Masashi Nakagawa - Web page: https://patentgrep.com/patent/US20180025533A1 ## CPC - G06T 15/30: Clipping - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - H04N 13/327: Calibration thereof - H04N 13/344: with head-mounted left-right displays - H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image processing apparatus includes an image acquisition unit acquiring a first and second captured images from first and second points of view respectively, an initial value acquisition unit acquiring initial values of respective clip positions to clip display images from the first and second captured images, a derivation unit deriving an amount of a first exterior region of a first display image outside a first region of the first captured image when the first display image is clipped based on the initial values, and deriving an amount of a second exterior region of a second display image outside a second region of the second captured image when the second display image is clipped based on the initial values, and a determination unit determining the respective clip positions to clip the display images from the first and second captured images based on the first and second amounts. ## Claims ### Claim 1 An image processing apparatus comprising: an image acquisition unit configured to acquire a first captured image captured from a first point of view and a second captured image captured from a second point of view which is different from the first point of view; an initial value acquisition unit configured to acquire initial values of respective clip positions to clip display images from the first captured image and the second captured image; a derivation unit configured to derive a first out-of-area amount that indicates an amount of a first exterior region of a first display image located outside a first region of the first captured image when the first display image is clipped from the first captured image based on the initial values and to derive a second out-of-area amount that indicates an amount of a second exterior region of a second display image located outside a second region of the second captured image when the second display image is clipped from the second captured image based on the initial values; and a determination unit configured to determine the respective clip positions to clip the display images from the first captured image and the second captured image based on the first and second out-of-area amounts. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the determination unit acquires a sum of the first out-of-area amount and the second out-of-area amount and determines the respective clip positions based on the sum. ### Claim 3 (depends on 2) The image processing apparatus according to claim 2, wherein the determination unit determines the respective clip positions so that the sum is minimized. ### Claim 4 (depends on 2) The image processing apparatus according to claim 2, wherein, when one of the clip positions to minimize the sum is greater than a threshold value, the determination unit determines the threshold value as that one of the clip positions. ### Claim 5 (depends on 1) The image processing apparatus according to claim 1, wherein the determination unit determines a correction amount to correct the initial values of the respective clip positions based on the first and second out-of-area amounts and determines an initial value corrected based on the correction amount as the clip position. ### Claim 6 (depends on 1) The image processing apparatus according to claim 1, further comprising: a clipping unit configured to clip the display images, from the image captured from the first point of view and the image captured from the second point of view, based on the determined respective clip positions; a generating unit configured to generate an image based on the display images clipped by the clipping unit and based on an image of a virtual object; and an output unit configured to output the generated image to a display. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein a first image capturing device that captures the first captured image and a second image capturing device that captures the second captured image are provided in a head-mounted display. ### Claim 8 An image processing apparatus comprising: an image acquisition unit configured to acquire a first captured image captured from a first view point and a second captured image captured from a second point of view which is different from the first point of view; an initial value acquisition unit configured to acquire initial values of respective clip positions to clip display images from the first captured image and the second captured image; and a determination unit configured to, in a case that a first mode is set, derive a first out-of-area amount that indicates an amount of a first exterior region of a first display image located outside a first region of the first captured image when the first display image is clipped from the first captured image based on the initial values, and derive a second out-of-area amount that indicates an amount of a second exterior region of a second display image located outside a second region of the second captured image when the second display image clipped from the second captured image based on the initial values and determine respective clip positions to clip the display images from the first captured image and the second captured image based on the derived amounts and, in a case that a second mode different from the first mode is set, determine the initial values as the respective clip positions to clip the display images from the first captured image and the second captured image. ### Claim 9 (depends on 8) The image processing apparatus according to claim 8, further comprising a setting unit configured to set one of the first mode and the second mode. ### Claim 10 (depends on 9) The image processing apparatus according to claim 9, wherein the setting unit sets the first mode or the second mode based on an input by a user. ### Claim 11 (depends on 8) The image processing apparatus according to claim 8, further comprising a head-mounted display that includes a first image capturing device that captures the first captured image and includes a second image capturing device that captures the second captured image. ### Claim 12 (depends on 8) The image processing apparatus according to claim 8, further comprising: a clipping unit configured to clip the display images, from the image captured from the first point of view and the image captured from the second point of view, based on the determined respective clip positions; a generating unit configured to generate an image based on the images clipped by the clipping unit and an image of virtual object; and an output unit configured to output the generated image to a display. ### Claim 13 An image processing method comprising: acquiring a first captured image captured from a first point of view and a second captured image captured from a second point of view which is different from the first point of view; acquiring initial values of respective clip positions to clip display images from the first captured image and the second captured image; deriving a first out-of-area amount that indicates an amount of a first exterior region of a first display image located outside a first region of the first captured image when the first display image is clipped from the first captured image based on the initial values, and deriving a second out-of-area amount that indicates an amount of a second exterior region of a second display image located outside a second region of the second captured image when the second display image is clipped from the second captured image based on the initial values; and determining the respective clip positions to clip the display images from the first captured image and the second captured image based on the first and second out-of-area amounts. ### Claim 14 An image processing method comprising: acquiring a first captured image captured from a first point of view and a second captured image captured from a second point of view which is different from the first point of view. acquiring initial values of respective clip positions to clip display images from the first captured image and the second captured image; and in a case that a first mode is set, deriving a first out-of-area amount that indicates an amount of a first exterior region of a first display image located outside a first region of the first captured image when the first display image is clipped from the first captured image based on the initial values, and deriving a second out-of-area amount that indicates an amount of a second exterior region of a second display image located outside a second region of the second captured image when the second display image is clipped from the second captured image based on the initial values, and determining respective clip positions to clip the display images from the first captured image and the second captured image based on the first and second out-of-area amounts and, in a case that a second mode which is different from the first mode is set, determining the initial values as the respective clip positions to clip the display images from the first captured image and the second captured image. ### Claim 15 A non-transitory computer-readable storage medium storing a program that when executed by a computer causes the computer to serve as: an image acquisition unit configured to acquire a first captured image captured from a first point of view and a second captured image captured from a second point of view which is different from the first point of view; an initial value acquisition unit configured to acquire initial values of respective clip positions to clip display images from the first captured image and the second captured image; a derivation unit configured to derive a first out-of-area amount that indicates an amount of a first exterior region of a first display image located outside the first captured image when the first display image is clapped from the first captured image based on the initial values, and to derive a second out-of-area amount that indicates an amount of a second exterior region of a second display image located outside a second exterior region of the second captured image when the second display image is clipped from the second captured image based on the initial values; and a determination unit configured to determine the respective clip positions to clip the display images from the first captured image and the second captured image based on the first and second out-of-area amounts. ### Claim 16 A non-transitory computer-readable storage medium storing a program that when executed by a computer causes the computer to serve as: an image acquisition unit configured to acquire a first captured image captured from a first point of view and a second captured image captured from a second point of view which is different from the first point of view; an acquisition unit configured to acquire initial values of respective clip positions to clip display images from the first captured image and the second captured image; and a determination unit configured to, in case that a first mode is set, derive a first out-of-area amount that indicates an amount of a first exterior region of a first display image located outside a first region of the first captured image when the first display image is clipped from the first captured image based on the initial values, and derive a second out-of-area amount that indicates an amount of a second exterior region of a second display image located outside a second region of the second captured image when the second display image clipped from the second captured image based on the initial values, and determine respective clip positions to clip the display images from the first captured image and the second captured image based on the derived amounts and, in a case that a second mode which is different from the first mode is set, determine the initial values as the respective clip positions to clip the display images from the first captured image and the second captured image.