# US 12,217,528 B2 · Image processing apparatus for determining an object area where part of the object is occluded > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,528 B2 - Document ID: US12217528B2 - Status: Granted - Kind: B2 - Application number: 17/695,622 - Filed: 2022-03-15 - Published: 2022-06-30 - Granted: 2025-02-04 - Earliest claimed: 2019-09-20 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Hiroyuki Uchiyama; Hiroshi Tojo; Shinji Yamamoto - Web page: https://patentgrep.com/patent/US12217528B2 ## CPC - G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition} - G06T 7/73: using feature-based methods - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/40: Extraction of image or video features - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 10/98: Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns - G06T 2207/20076: Probabilistic image processing - G06T 2207/20081: Training; Learning - G06T 2207/30196: Human being; Person - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image processing apparatus includes a first detection unit configured to detect, from an image in which an object including a plurality of parts is captured, first feature points corresponding to the parts of the object, an acquisition unit configured to acquire a reliability indicating a likelihood that a position indicated by a feature point is a part corresponding to the feature point for each of the first feature points detected by the first detection unit, a second detection unit configured to detect a second feature point based on some of the first feature points for a part corresponding to a first feature point with the low reliability, and a determination unit configured to determine an area including the object based on some of the first feature points and the second feature point. ## Claims ### Claim 1 An image processing apparatus comprising: at least one memory storing instructions; and at least one processor that, upon execution of the stored instructions, are configured to: detect, from an image in which an object including a plurality of parts is captured, first feature points corresponding to the parts of the object; acquire a reliability indicating a likelihood that a position indicated by a feature point is a part corresponding to the feature point for each of the first feature points detected; detect a second feature point based on some of the first feature points for a part corresponding to a first feature point with the low reliability; and determine an area including the object based on some of the first feature points and the second feature point. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the second feature point detected is obtained by correcting a position of a part indicated by the first feature point whose reliability is lower than a predetermined value among the first feature points based on the first feature point indicating a position of a part close to the part indicated by the first feature point whose reliability is lower than the predetermined value among the first feature points. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the second feature point detected is obtained by correcting a position of a part indicated by the first feature point whose reliability is lower than a predetermined value among the first feature points based on the first feature point whose reliability is higher than the predetermined value among the first feature points. ### Claim 4 (depends on 1) The image processing apparatus according to claim 1, wherein the second feature point obtained by correcting a position of a part indicated by the first feature point with the low reliability among the first feature points based on a positional relationship among the plurality of parts included in the object. ### Claim 5 (depends on 4) The image processing apparatus according to claim 4, wherein positions of a head, a neck, a waist, and an ankle of a person is detected as the first feature points, and wherein, in a case where the reliability of the position of the ankle is low, the position of the ankle is detected as the second feature point based on a positional relationship between one of the head and the waist of the person and the ankle of the person. ### Claim 6 (depends on 1) The image processing apparatus according to claim 1, wherein the second feature point is detected for the part indicated by the first feature point whose reliability is lower than the predetermined value among the first feature points detected, based on the first feature point indicating the part and whose reliability is higher than a predetermined value in a previous image. ### Claim 7 (depends on 6) The image processing apparatus according to claim 6, wherein positions of a head, a neck, a waist, and an ankle of a person are detected as the first feature points from the image, wherein the reliability of the position of the ankle of the person is acquired in a previous image captured before the image is captured, and wherein, in a case where the reliability of the position of the ankle of the person in the previous image is higher than the predetermined value, the position of the ankle of the person in the previous image is detected as the second feature point in the image. ### Claim 8 (depends on 1) The image processing apparatus according to claim 1, wherein the object is a person, and wherein the first feature points are detected by inputting the image to a trained model obtained by learning of feature points corresponding to the parts of the person as the first feature points of the person. ### Claim 9 (depends on 1) The image processing apparatus according to claim 1, wherein the at least one processor is further configured to: extract a feature amount for recognizing the object based on a partial image obtained by clipping the area determined from the image; and recognize whether the object captured in the image is identical to a specific object being preliminarily registered, by comparing the extracted feature amount with a feature amount of the specific object. ### Claim 10 (depends on 9) The image processing apparatus according to claim 9, wherein the object is a human body, and wherein a person captured in the image is identified from among preliminarily registered persons by comparing the extracted feature amount with feature amounts of the preliminarily registered persons. ### Claim 11 (depends on 10) The image processing apparatus according to claim 10, wherein the feature amount is extracted of a partial image obtained by clipping the area determined from the image, based on a trained model for outputting a feature amount indicating each part of the object from an input image. ### Claim 12 (depends on 10) The image processing apparatus according to claim 10, wherein a feature amount is extracted from a partial area corresponding to the part of which the reliability of the first feature point is higher in a partial image obtained by clipping the area determined from the image. ### Claim 13 (depends on 1) The image processing apparatus according to claim 1, wherein the at least one processor is further configured to integrate the feature amount extracted from the image for each of the parts based on the reliability. ### Claim 14 (depends on 1) The image processing apparatus according to claim 1, wherein, in a case where a feature point is located outside a predetermined area, the reliability of the feature point is acquired in such a manner that the reliability is low. ### Claim 15 (depends on 1) The image processing apparatus according to claim 1, wherein an area including the object is determined based on a central axis of the object estimated based on the detected first feature points. ### Claim 16 (depends on 1) The image processing apparatus according to claim 1, wherein a rectangle including some of the first feature points is determined as the area including the object. ### Claim 17 (depends on 1) The image processing apparatus according to claim 1, wherein the at least one processor is further configured to output the first feature points detected and the second feature point detected in a distinguishable manner. ### Claim 18 An image processing apparatus comprising: at least one memory storing instructions; and at least one processor that, upon execution of the stored instructions, are configured to: acquire a feature point indicating a position in an image of each of a plurality of parts of an object detected from the image, and a reliability indicating a likelihood that a part correspond to the feature point; extract a first image feature from the image for each of the parts indicated by the feature points; output a second image feature indicating a feature amount specific to the object based on the reliability and the first image feature; and identify the object detected from the image and a preliminarily registered predetermined object based on a feature amount indicating the preliminarily registered predetermined object and the second feature amount. ### Claim 19 An image processing method comprising: detecting, as a first detection, from an image in which an object including a plurality of parts is captured, first feature points corresponding to the parts of the object; acquiring a reliability indicating a likelihood that a position indicated by a feature point is a part corresponding to the feature point for each of the first feature points detected in the first detection; detecting, as a second detection, a second feature point based on some of the first feature points for a part corresponding to the first feature point with the low reliability; and determining an area including the object based on some of the first feature points and the second feature point. ### Claim 20 A non-transitory computer-readable storage medium storing a program for causing a computer to execute an image processing method, the image processing method comprising: detecting, as a first detection, from an image in which an object including a plurality of parts is captured, first feature points corresponding to the parts of the object; acquiring a reliability indicating a likelihood that a position indicated by a feature point is a part corresponding to the feature point for each of the first feature points detected in the first detection; detecting, as a second detection, a second feature point based on some of the first feature points for a part corresponding to the first feature point with the low reliability; and determining an area including the object based on some of the first feature points and the second feature point.