# US 12,010,437 B2 · Arrangement determination apparatus, system, arrangement determination method, and recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,437 B2 - Document ID: US12010437B2 - Status: Granted - Kind: B2 - Application number: 17/518,444 - Filed: 2021-11-03 - Published: 2022-02-24 - Granted: 2024-06-11 - Earliest claimed: 2019-05-16 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Michio Aizawa; Yasushi Shikata; Shinichiro Uno; Atsushi Date - Web page: https://patentgrep.com/patent/US12010437B2 ## CPC - H04N 13/117: the virtual viewpoint locations being selected by the viewers or determined by viewer tracking - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N 5/265: Mixing - H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems - H04N 13/296: Synchronisation thereof; Control thereof - H04N 23/61: based on recognised objects - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An arrangement determination apparatus for determining an arrangement of a plurality of cameras that is used for generation of a virtual viewpoint image acquires information indicating an imaging target region, and determines an arrangement of the plurality of cameras based on the information in such a manner that images of the entire region of the imaging target region are captured by two or more cameras of the plurality of cameras and images of the entire region of the imaging target region are captured by different two or more cameras of the plurality of cameras that are different from the two or more cameras. ## Claims ### Claim 1 An arrangement determination apparatus comprising: one or more memories storing instructions; and one or more processors executing the instructions to: obtain first information representing an imaging target region captured by a plurality of imaging apparatuses used to generate a virtual viewpoint image; obtain second information regarding a plurality of groups corresponding to a plurality of directions in which the imaging target region is to be captured; and determine an arrangement of the plurality of imaging apparatuses based on the obtained first information and the obtained second information, wherein the imaging target region is captured from a first direction by two or more imaging apparatuses of the plurality of imaging apparatuses that is arranged according to the determined arrangement, the two or more imaging apparatuses belonging to a first group corresponding to the first direction among the plurality of groups, and the imaging target region is captured from a second direction different from the first direction by other two or more imaging apparatuses of the plurality of imaging apparatuses that are different from the two or more imaging apparatuses, the other two or more imaging apparatuses belonging to a second group corresponding to the second direction among the plurality of groups. ### Claim 2 (depends on 1) The arrangement determination apparatus according to claim 1, wherein the one or more processors further execute the instructions to obtain third information relating to resolution of an image captured by an imaging apparatus, and wherein the third information represents, in a case where a reference object is in the imaging target region, an allowable value of the number of pixels representing a size of the reference object in an image acquired by the imaging apparatuses performing image capturing. ### Claim 3 (depends on 2) The arrangement determination apparatus according to claim 2, wherein the reference object is an object having a predetermined size with respect to the imaging target region. ### Claim 4 (depends on 2) The arrangement determination apparatus according to claim 2, wherein the reference object is an upstanding person having a predetermined body height. ### Claim 5 (depends on 2) The arrangement determination apparatus according to claim 2, wherein the allowable value is determined based on an intended purpose of the virtual viewpoint image. ### Claim 6 (depends on 5) The arrangement determination apparatus according to claim 5, wherein, in a case where the virtual viewpoint image is used in live broadcast, the allowable value is smaller than that in a case where the virtual viewpoint image is used in archive distribution. ### Claim 7 (depends on 1) The arrangement determination apparatus according to claim 1, wherein the one or more processors further execute the instruction to obtain third information representing the number of the imaging apparatuses, wherein an arrangement of the plurality of imaging apparatuses is determined further based on the third information. ### Claim 8 (depends on 7) The arrangement determination apparatus according to claim 7, wherein the number is determined based on processing performance of an apparatus to generate the virtual viewpoint image. ### Claim 9 (depends on 1) The arrangement determination apparatus according to claim 1, wherein an arrangement of the plurality of imaging apparatuses is determined further based on third information regarding a site in which the plurality of imaging apparatuses is arranged. ### Claim 10 (depends on 1) The arrangement determination apparatus according to claim 1, wherein the second information is information representing the number of groups corresponding to the plurality of directions, wherein arrangement regions of the groups are determined based on the obtained second information, and wherein for each of the groups, an arrangement of imaging apparatuses belonging to the groups is determined. ### Claim 11 (depends on 10) The arrangement determination apparatus according to claim 10, wherein the number of groups is determined based on a type of an event. ### Claim 12 (depends on 1) The arrangement determination apparatus according to claim 1, wherein an angle formed by line segments connecting a predetermined position in the imaging target region and arrangement regions of respective adjacent two groups does not exceed a predetermined value. ### Claim 13 (depends on 1) The arrangement determination apparatus according to claim 1, wherein an interval between imaging apparatuses in a group does not exceed a predetermined distance. ### Claim 14 (depends on 1) The arrangement determination apparatus according to claim 1, wherein an imaging apparatus of the two or more imaging apparatuses that is arranged according to the determined arrangement and that is closer to the other two or more imaging apparatuses is closer to a different imaging apparatus of the two or more imaging apparatuses than all imaging apparatuses of the other two or more imaging apparatuses. ### Claim 15 (depends on 1) The arrangement determination apparatus according to claim 1, wherein the imaging target region is at least a part of a space serving as a target of image capturing to be performed by the plurality of imaging apparatuses. ### Claim 16 (depends on 15) The arrangement determination apparatus according to claim 15, wherein the space includes a first imaging target region, and a second imaging target region different from the first imaging target region, wherein the first imaging target region is captured by two or more first imaging apparatuses of the plurality of imaging apparatuses that is arranged according to the determined arrangement and the first imaging target region is captured by two or more second imaging apparatuses of the plurality of imaging apparatuses that are different from the first imaging apparatuses, and wherein the second imaging target region is captured by two or more third imaging apparatuses of the plurality of imaging apparatuses that are different from the first and second imaging apparatuses, and the first imaging target region is captured by two or more fourth imaging apparatuses of the plurality of imaging apparatuses that are different from the first to third imaging apparatuses. ### Claim 17 (depends on 16) The arrangement determination apparatus according to claim 16, wherein the second imaging target region includes the first imaging target region. ### Claim 18 A system comprising: a plurality of imaging apparatuses configured to capture an imaging target region, and a generation apparatus configured to generate a virtual viewpoint image based on a captured image acquired by the plurality of imaging apparatuses, wherein the plurality of imaging apparatuses comprises two or more imaging apparatuses that capture the imaging target region from a first direction and that belong to a first group which is included in a plurality of groups corresponding to a plurality of directions and which corresponds to the first direction included in the plurality of directions, and different two or more imaging apparatuses of the plurality of imaging apparatuses that are different from the two or more imaging apparatuses, that capture the imaging target region from a second direction, and that belong to a second group which is included in the plurality of groups corresponding to the plurality of directions and which corresponds to the second direction of the plurality of directions. ### Claim 19 (depends on 18) The system according to claim 18, wherein an imaging apparatus of the two or more imaging apparatuses that is closer to the other two or more imaging apparatuses is closer to a different imaging apparatus of the two or more imaging apparatuses than all imaging apparatuses of the other two or more imaging apparatuses. ### Claim 20 (depends on 18) The system according to claim 18, wherein an angle formed by line segments connecting a predetermined position in the imaging target region and arrangement regions of respective adjacent two groups does not exceed a predetermined value. ### Claim 21 (depends on 18) The system according to claim 18, wherein an angle formed by the first direction and the second direction does not exceed a predetermined value. ### Claim 22 An arrangement determination method comprising: obtaining first information representing an imaging target region captured by a plurality of imaging apparatuses used to generate a virtual viewpoint image; obtaining second information regarding a plurality of groups corresponding to a plurality of directions in which the imaging target region is to be captured; and determining an arrangement of the plurality of imaging apparatuses based on the obtained first information and the obtained second information wherein the imaging target region is captured from a first direction by two or more imaging apparatuses of the plurality of imaging apparatuses that is arranged according to the determined arrangement, the two or more imaging apparatuses belonging to a first group corresponding to the first direction among the plurality of groups, the imaging target region is captured from a second direction different from the first direction by different two or more imaging apparatuses of the plurality of imaging apparatuses that are different from the two or more imaging apparatuses, the other two or more imaging apparatuses belonging to a second group corresponding to the second direction among the plurality of groups. ### Claim 23 A non-transitory computer-readable recording medium storing a program that causes a computer to execute an arrangement determination method comprising: obtaining first information representing an imaging target region captured by a plurality of imaging apparatuses used to generate a virtual viewpoint image; obtaining second information regarding a plurality of groups corresponding to a plurality of directions in which the imaging target region is to be captured; and determining an arrangement of the plurality of imaging apparatuses based on the obtained first information and the obtained second information wherein the imaging target region is captured from a first direction by two or more imaging apparatuses of the plurality of imaging apparatuses that is arranged according to the determined arrangement, the two or more imaging apparatuses belonging to a first group corresponding to the first direction among the plurality of groups, the imaging target region is captured from a second direction different from the first direction by different two or more imaging apparatuses of the plurality of imaging apparatuses that are different from the two or more imaging apparatuses, the other two or more imaging apparatuses belonging to a second group corresponding to the second direction among the plurality of groups.