# US 2021/0241541 A1 · Information processing apparatus, information processing method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241541 A1 - Document ID: US20210241541A1 - Status: Published application - Kind: A1 - Application number: 17/166,681 - Filed: 2021-02-03 - Published: 2021-08-05 - Earliest claimed: 2020-02-05 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Nozomu Kasuya - Web page: https://patentgrep.com/patent/US20210241541A1 ## CPC - G06T 7/85: {Stereo camera calibration} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 7/593: from stereo images - G06T 7/73: using feature-based methods - H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation - H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems - H04N 23/683: {performed by a processor, e.g. controlling the readout of an image memory} - G06T 2200/08: involving all processing steps from image acquisition to 3D model generation - G06T 2207/10012: Stereo images - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - G06T 2207/10148: Varying focus - G06T 2207/30196: Human being; Person - G06T 2207/30221: Sports video; Sports image - G06T 2207/30228: Playing field - G06T 2219/2016: Rotation, translation, scaling - G06T 2219/2021: Shape modification - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An information processing apparatus comprising: a correction unit configured to correct a shift between a first three-dimensional position in an image capturing region that is identified based on a position of a first feature point on a first image among a plurality of images acquired by a first acquisition unit and image capturing information about an image capturing apparatus that corresponds to the first image and a second three-dimensional position in an image capturing region that is identified based on a position of a second feature point corresponding to the first feature point on a second image among the plurality of images acquired by the first acquisition unit and the image capturing information about the image capturing apparatus that corresponds to the second image. ## Claims ### Claim 1 An information processing apparatus comprising: a first acquisition unit configured to acquire a plurality of images to be used in generating image content based on capturing an image of an image capturing region at a predetermined time point from different directions by a plurality of image capturing apparatuses; a second acquisition unit configured to acquire a plurality of pieces of image capturing information including information from which at least one of a position and an orientation of the plurality of image capturing apparatuses corresponding to the plurality of images acquired by the first acquisition unit is to be identified; and a correction unit configured to correct a shift between a first three-dimensional position in the image capturing region that is identified based on a position of a first feature point on a first image among the plurality of images acquired by the first acquisition unit and the image capturing information about the image capturing apparatus that corresponds to the first image and a second three-dimensional position in the image capturing region that is identified based on a position of a second feature point corresponding to the first feature point on a second image among the plurality of images acquired by the first acquisition unit and the image capturing information about the image capturing apparatus that corresponds to the second image. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the correction unit corrects the shift based on the position of the second feature point on the second image and a position on the second image that is identified by projecting the first feature point onto the second image based on the image capturing information about the image capturing apparatus that corresponds to the first image and the image capturing information about the image capturing apparatus that corresponds to the second image. ### Claim 3 (depends on 1) The information processing apparatus according to claim 1, wherein the correction unit corrects the shift based on the first three-dimensional position, the position of the second feature point on the second image, and the image capturing information about the image capturing apparatus that corresponds to the second image. ### Claim 4 (depends on 1) The information processing apparatus according to claim 1, wherein the correction unit corrects the shift by modifying the second image. ### Claim 5 (depends on 1) The information processing apparatus according to claim 1, wherein the correction unit corrects the shift by changing the image capturing information about the image capturing apparatus that corresponds to the second image. ### Claim 6 (depends on 1) The information processing apparatus according to claim 1, wherein the plurality of images acquired by the first acquisition unit includes an image acquired by performing correction processing to correct a shift between a position of a third feature point on a third image acquired based on capturing an image of the image capturing region at the predetermined time point by the image capturing apparatus and a position of a fourth feature point, corresponding to the third feature point, on a fourth image acquired based on capturing an image of the image capturing region at another time point different from the predetermined time point by the image capturing apparatus. ### Claim 7 (depends on 6) The information processing apparatus according to claim 6, wherein the first acquisition unit further acquires information indicating reliability of the correction processing. ### Claim 8 (depends on 7) The information processing apparatus according to claim 7, wherein the correction unit selects the first image from among the plurality of images acquired by the first acquisition unit, based on the information indicating reliability of the correction processing. ### Claim 9 (depends on 8) The information processing apparatus according to claim 8, wherein, based on the information indicating reliability of the correction processing acquired by the first acquisition unit, the correction unit selects an image corresponding to information indicating reliability of the correction processing that is higher than a predetermined threshold value, as the first image from among the plurality of images acquired by the first acquisition unit. ### Claim 10 (depends on 8) The information processing apparatus according to claim 8, wherein, based on the information indicating reliability of the correction processing acquired by the first acquisition unit, the correction unit selects an image corresponding to information indicating reliability of the correction processing that is higher than reliability of the correction processing on another image acquired by performing the correction processing, as the first image from among the plurality of images acquired by the first acquisition unit. ### Claim 11 (depends on 7) The information processing apparatus according to claim 7, wherein the information indicating reliability of the correction processing indicates a first reliability in a case where a difference between a position of a feature point on an image that is acquired by performing the correction processing and the position of the fourth feature point on the fourth image is a first value, and wherein the information indicating reliability of the correction processing indicates a second reliability higher than the first reliability in a case where the difference between the position of the feature point on the image that is acquired by performing the correction processing and the position of the fourth feature point on the fourth image is a second value less than the first value. ### Claim 12 (depends on 1) The information processing apparatus according to claim 1, wherein a focal length of the image capturing apparatus that corresponds to the first image is less than a focal length of the image capturing apparatus that corresponds to the second image. ### Claim 13 (depends on 1) The information processing apparatus according to claim 1, wherein the second feature point is a feature point having the same feature as a feature of the first feature point in a case where the first image and the second image are projected onto the same two-dimensional coordinate based on the image capturing information about the image capturing apparatus that corresponds to the first image and the image capturing information about the image capturing apparatus that corresponds to the second image. ### Claim 14 (depends on 1) The information processing apparatus according to claim 1, wherein the correction unit corrects the shift in a case where at least one of the position and the orientation of the image capturing apparatus is changed by a shake of the image capturing apparatus. ### Claim 15 (depends on 1) The information processing apparatus according to claim 1, further comprising a generation unit configured to generate shape data contained in the image content and representing a shape of an object in the image capturing region, based on the plurality of images acquired by the first acquisition unit and the plurality of pieces of image capturing information acquired by the second acquisition unit. ### Claim 16 (depends on 15) The information processing apparatus according to claim 15, wherein the correction unit modifies the second image, and wherein the generation unit generates the shape data based on the plurality of images including the second image modified by the correction unit and the plurality of pieces of image capturing information acquired by the second acquisition unit. ### Claim 17 (depends on 16) The information processing apparatus according to claim 16, wherein the correction unit changes the image capturing information about the image capturing apparatus that corresponds to the second image, and wherein the generation unit generates the shape data based on the plurality of images acquired by the first acquisition unit and the plurality of pieces of image capturing information including the image capturing information that is changed by the correction unit and is the image capturing information about the image capturing apparatus that corresponds to the second image. ### Claim 18 (depends on 1) The information processing apparatus according to claim 1, wherein each of the first feature point and the second feature point is a feature point based on an object that changes in position at a different time point in the image capturing region. ### Claim 19 An information processing method comprising: acquiring, as first acquiring, a plurality of images to be used in generating image content based on capturing an image of an image capturing region at a predetermined time point from different directions by a plurality of image capturing apparatuses; acquiring, as second acquiring, a plurality of pieces of image capturing information including information from which at least one of a position and an orientation of the plurality of image capturing apparatuses corresponding to the plurality of acquired images is to be identified; and correcting a shift between a first three-dimensional position in the image capturing region that is identified based on a position of a first feature point on a first image among the plurality of images acquired by the first acquiring and the image capturing information about the image capturing apparatus that corresponds to the first image and a second three-dimensional position in the image capturing region that is identified based on a position of a second feature point corresponding to the first feature point on a second image among the plurality of images acquired by the first acquiring and the image capturing information about the image capturing apparatus that corresponds to the second image. ### Claim 20 A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method, the method comprising: acquiring, as first acquiring, a plurality of images to be used in generating image content based on capturing an image of an image capturing region at a predetermined time point from different directions by a plurality of image capturing apparatuses; acquiring, as second acquiring, a plurality of pieces of image capturing information including information from which at least one of a position and an orientation of the plurality of image capturing apparatuses corresponding to the plurality of acquired images is to be identified; and correcting a shift between a first three-dimensional position in the image capturing region that is identified based on a position of a first feature point on a first image among the plurality of images acquired by the first acquiring and the image capturing information about the image capturing apparatus that corresponds to the first image and a second three-dimensional position in the image capturing region that is identified based on a position of a second feature point corresponding to the first feature point on a second image among the plurality of images acquired by the first acquiring and the image capturing information about the image capturing apparatus that corresponds to the second image.