# US 2022/0044477 A1 · Generation apparatus, generation method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0044477 A1 - Document ID: US20220044477A1 - Status: Published application - Kind: A1 - Application number: 17/387,295 - Filed: 2021-07-28 - Published: 2022-02-10 - Earliest claimed: 2020-08-05 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Yasufumi Takama; Hidenori Ito - Web page: https://patentgrep.com/patent/US20220044477A1 ## CPC - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06T 7/564: from contours - G06T 7/75: {involving models} - G06T 15/205: {Image-based rendering} - G06T 2207/20076: Probabilistic image processing - G06T 2207/20224: Image subtraction - G06T 2207/30196: Human being; Person - G06T 2207/30221: Sports video; Sports image - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A pose estimation apparatus (110) obtains a plurality of images captured by a plurality of image capturing apparatuses (100) from different directions, specifies an image that is to be used for generating a three-dimensional pose model indicating a plurality of joint positions of an object, from the plurality of images, and generates a three-dimensional pose model of the object based on the specified image. ## Claims ### Claim 1 A generation apparatus comprising: one or more memories storing instructions; and one or more processors that, upon executing the instructions, performs: obtaining a plurality of images captured by a plurality of image capturing apparatuses from different directions; specifying an image that is to be used for generating a three-dimensional pose model indicating a plurality of joint positions of an object, from the obtained plurality of images; and generating a three-dimensional pose model of the object based on specified image. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the one or more processors further execute the instructions to perform obtaining a three-dimensional shape model indicating a three-dimensional shape of the object, based on the plurality of images, wherein, in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified based on the obtained shape model. ### Claim 3 (depends on 2) The apparatus according to claim 2, wherein, in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified based on the three-dimensional shape model of the object and a region of the object in each of the plurality of images. ### Claim 4 (depends on 3) The apparatus according to claim 3, wherein, in the specifying, priority orders for the plurality of images are set based on the three-dimensional shape model of the object and the region of the object in each of the plurality of images, and an image that is to be used for generating the three-dimensional pose model of the object is specified in accordance with the priority orders. ### Claim 5 (depends on 1) The apparatus according to claim 1, wherein, in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified based on an image resolution of a region of the object in each of the plurality of images. ### Claim 6 (depends on 1) The apparatus according to claim 1, wherein, in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified based on a size of a region of the object in each of the plurality of images. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein, in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified based on positions of the plurality of image capturing apparatuses. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the one or more processors further execute the instructions to perform determining whether or not at least a portion of a region of the object in each of the plurality of images is occluded by a region of another object from among a plurality of objects, wherein, in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified based on a result of the determination made by the determining unit. ### Claim 9 (depends on 8) The apparatus according to claim 8, wherein in the determining, it is determined whether or not at least a portion of the region of the object is occluded by the region of the other object, based on a distance between the object and an image capturing apparatus that captures an image of the object from among the plurality of image capturing apparatuses, and the distance between the image capturing apparatus and the other object. ### Claim 10 (depends on 1) The apparatus according to claim 1, wherein the one or more processors further execute the instructions to perform acquiring a reliability indicating accuracy of the object regarding the plurality of images, wherein, in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified based on the reliability obtained by the reliability obtaining unit. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein, in the obtaining, an image indicating a region of the object is obtained for each of the plurality of captured images based on a probability value indicating whether or not each pixel in the captured image is a pixel that constitutes the region of the object. ### Claim 13 (depends on 11) The apparatus according to claim 11, wherein in the acquiring, the reliability for an image indicating the region of the object is acquired based on the probability value. ### Claim 14 (depends on 13) The apparatus according to claim 13, wherein, in the acquiring the reliability is set such that, in a distribution of the probability value, the higher a ratio of probability values that are close to a minimum value and a maximum value that a probability value possibly takes is, the higher a value of the reliability becomes. ### Claim 15 (depends on 10) The apparatus according to claim 10, wherein, in the obtaining, an image indicating the object is obtained based on a difference value between a pixel value of the captured image and a pixel value of a background image in which the object is not present in a shooting region corresponding to a captured image, for each of a plurality of captured images. ### Claim 16 (depends on 15) The apparatus according to claim 15, wherein, in the acquiring, the reliability for an image indicating the region of the object is acquired based on the difference value. ### Claim 17 (depends on 11) The apparatus according to claim 11, wherein, in the obtaining, the probability value of each pixel in the captured images is obtained based on a result obtained through learning. ### Claim 17 (depends on 16) The apparatus according to claim 16, wherein, in the acquiring, the reliability is acquired such that the larger a value of the difference value in the captured image is, the larger a value of the reliability becomes. ### Claim 18 (depends on 1) The apparatus according to claim 1, wherein the plurality of image capturing apparatuses are image capturing apparatuses that are used for generating a virtual viewpoint image, and in the specifying, an image that is to be used for generating the three-dimensional pose model of the object is specified from a plurality of images captured and obtained by the plurality of image capturing apparatuses that are used for generating the virtual viewpoint image. ### Claim 19 A method for generating a three-dimensional pose model, the method comprising: obtaining a plurality of images captured by a plurality of image capturing apparatuses from different directions; specifying an image that is to be used for generating a three-dimensional pose model indicating a plurality of joint positions of an object, from the plurality of images; and generating a three-dimensional pose model of the object based on the specified image. ### Claim 20 A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method comprising: obtaining a plurality of images captured by a plurality of image capturing apparatuses from different directions; specifying an image that is to be used for generating a three-dimensional pose model indicating a plurality of joint positions of an object, from the plurality of images; and generating a three-dimensional pose model of the object based on the specified image.