# US 2018/0322651 A1 · Image processing device and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322651 A1 - Document ID: US20180322651A1 - Status: Published application - Kind: A1 - Application number: 15/772,935 - Filed: 2016-09-29 - Published: 2018-11-08 - Earliest claimed: 2015-11-20 - Assignees: Sony Corporation; Sony Interactive Entertainment Inc. - Applicants: Sony Corporation; Sony Interactive Entertainment Inc. - Inventors: Atsushi Kimura; Daisuke Tahara; Hiroyuki Segawa; Tetsugo Inada; Akio Ohba; Hirofumi Okamoto - Web page: https://patentgrep.com/patent/US20180322651A1 ## CPC - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/60: Analysis of geometric attributes - G06T 7/75: {involving models} - G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition} - G06T 2207/30196: Human being; Person - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract [Object] To perform more stable and highly accurate attitude estimation. [Solution] The attitude optimization unit optimizes the articulation position, the angle, the number of articulations, and the like which are attitude parameters of a human body model (tree structure) by a plurality of optimization techniques so as to match a region in which a human body can exist, and switches among a plurality of optimization techniques and uses an optimum technique. Note that optimization techniques include 1. initial value, 2. algorithm, and 3, restriction, and optimization is performed by switching among these three. For example, it is possible to apply the present disclosure to an image processing device that performs image processing of optimizing the articulation position and angle of a human body model. ## Claims ### Claim 1 An image processing device comprising: a first attitude optimization unit configured to, with respect to a human body region extracted from an input image, optimize an attitude parameter of a human body model by using a first optimization technique; a second attitude optimization unit configured to, with respect to the human body region, optimize the attitude parameter of the human body model by using a second optimization technique different from the first optimization technique; and a selection unit configured to select one of a first optimization result obtained by the first optimization technique and a second optimization result obtained by the second optimization technique. ### Claim 2 (depends on 1) The image processing device according to claim 1, further comprising: an object detection unit configured to detect an object from the input image, wherein the selection unit selects one of the first optimization result and the second optimization result in accordance with an object detection result obtained by the object detection unit. ### Claim 3 (depends on 1) The image processing device according to claim 1, further comprising: a degree-of-reliability evaluation unit configured to evaluate a degree of reliability of the first optimization result and a degree of reliability of the second optimization result, wherein the selection unit selects one of the first optimization result and the second optimization result in accordance with a degree-of-reliability evaluation result obtained by the degree-of-reliability evaluation unit. ### Claim 4 (depends on 3) The image processing device according to claim 3, wherein in a case where the degrees of reliability of both of the optimization results are lower than a threshold value, the selection unit selects an attitude parameter in an initial state. ### Claim 5 (depends on 3) The image processing device according to claim 3, wherein in a case where the degrees of reliability of both of the optimization results are lower than a threshold value, the selection unit suspends processing using the first optimization technique and the second optimization technique, and prompts a user to perform calibration. ### Claim 6 (depends on 1) The image processing device according to claim 1, wherein the first optimization technique and the second optimization technique relate to at least one of an algorithm, an initial value, and a restriction condition. ### Claim 7 (depends on 1) The image processing device according to claim 1, wherein one of the first optimization technique and the second optimization technique is a technique based on an extremum seeking algorithm. ### Claim 8 (depends on 1) The image processing device according to claim 1, wherein one of the first optimization technique and the second optimization technique is a technique based on a specific attitude or a specific action. ### Claim 9 (depends on 1) The image processing device according to claim 1, further comprising: an object detection unit configured to detect an object from the input image, wherein one of the first optimization technique and the second optimization technique performs optimization using an optimization result in a previous frame as an initial value, and the other of the first optimization technique and the second optimization technique performs optimization using an initial value determined on a basis of an object detection result obtained by the object detection unit. ### Claim 10 (depends on 1) The image processing device according to claim 1, further comprising: an object detection unit configured to detect an object from the input image, wherein one of the first optimization technique and the second optimization technique performs optimization using a restriction determined on a basis of an object detection result obtained by the object detection unit. ### Claim 11 (depends on 1) The image processing device according to claim 1, wherein the attitude parameter of the human body model includes an articulation position, an angle, and a number of articulations. ### Claim 12 An image processing image method comprising, by a processing device: optimizing an attitude parameter of a human body model by using a first optimization technique, with respect to a human body region extracted from an input image; optimizing the attitude parameter of the human body model by using a second optimization technique different from the first optimization technique, with respect to the human body region; and selecting one of a first optimization result obtained by the first optimization technique and a second optimization result obtained by the second optimization technique.