# US 12,602,904 B2 · Method and electronic device for recognizing object based on mask updates > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,904 B2 - Document ID: US12602904B2 - Status: Granted - Kind: B2 - Application number: 18/517,809 - Filed: 2023-11-22 - Published: 2024-05-30 - Granted: 2026-04-14 - Earliest claimed: 2022-11-29 - Assignees: Electronics and Telecommunications Research Institute - Applicants: Electronics and Telecommunications Research Institute - Inventors: Ye-Bin Moon; Tae-Hyun Oh; Yongjin Kwon - Web page: https://patentgrep.com/patent/US12602904B2 ## CPC - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 10/764: using classification, e.g. of video objects - G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods} - G06V 10/82: using neural networks - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method of recognizing an object based on mask updates is provided. The method includes generating first result information in which a foreground and background of a predetermined input image have been separated from each other, generating second result information in which the results of the object recognition within the input image and a boundary portion of the object have been matched, calculating an objective function to be applied to each mask head based on the first and second result information and parameter information for the first and second result information, updating all of the mask heads based on the objective function, and performing object recognition based on the updated mask. ## Claims ### Claim 1 A method of recognizing an object based on mask updates, the method being performed by a computer and comprising: generating first result information in which a foreground and background of a predetermined input image have been separated from each other; generating second result information in which results of an object recognition within the input image and a boundary portion of the object have been matched; calculating an objective function to be applied to each of mask heads based on the first and second result information and parameter information for the first and second result information; updating all of the mask heads based on the objective function; and performing object recognition based on the updated mask heads, wherein the generating of the first result information comprises: calculating first similarity between a feature vector that is obtained by passing through a first layer of a mask head and a foreground prototype in each pixel of the input image; calculating second similarity between the feature vector and a background prototype; and generating the first result information by applying initial results of object recognition in each pixel of the input image to the first and second similarities. ### Claim 2 (depends on 1) The method of claim 1, wherein the calculating of the second similarity between the feature vector and the background prototype comprises: calculating a foreground error of a pixel that is located in a circumference of a bounding box of each object; extracting a set of pixels having a predetermined number in order that the calculated foreground error is high; and calculating the background prototype based on a feature vector of the set of extracted pixels. ### Claim 3 (depends on 1) The method of claim 1, wherein the generating of the first result information by applying the initial results of object recognition in each pixel of the input image to the first and second similarities comprises applying information on a set of pixels in which a minimum value between a foreground error and a background error is greater than a predetermined threshold to the calculated first similarity. ### Claim 4 (depends on 1) The method of claim 1, wherein the generating of the second result information in which the results of object recognition within the input image and the boundary portion of the object have been matched comprises: generating a set of pixels that neighbor a predetermined pixel that enters the mask head as an input; calculating third similarity between the predetermined pixel and neighboring pixels within the generated set of pixels; and matching the results of object recognition and the boundary portion based on whether the calculated third similarity is greater than a predetermined threshold. ### Claim 5 An electronic device for recognizing an object, comprising: a communication unit configured to receive a predetermined input image; memory in which a program for updating a mask based on a preset objective function with respect to the predetermined input image is stored; and a processor configured to generate first result information in which a foreground and background of the input image have been separated from each other, generate second result information in which results of an object recognition within the input image and a boundary portion of the object have been matched, calculate an objective function to be applied to each of mask heads based on the first and second result information and parameter information for the first and second result information, update all of the mask heads, and perform object recognition based on the updated mask heads, by executing the program stored in the memory, wherein the processor calculates first similarity between a feature vector that is obtained by passing through a first layer of a mask head and a foreground prototype in each pixel of the input image, calculates second similarity between the feature vector and a background prototype, and generates the first result information by applying initial results of object recognition in each pixel of the input image to the first and second similarities. ### Claim 6 (depends on 5) The electronic device of claim 5, wherein the processor calculates a foreground error of a pixel that is located in a circumference of a bounding box of each object included in the image, extracts a set of pixels having a predetermined number in order that the calculated foreground error is high, and calculates the background prototype based on a feature vector of the set of extracted pixels. ### Claim 7 (depends on 5) The electronic device of claim 5, wherein the processor applies information on a set of pixels in which a minimum value between a foreground error and a background error is greater than a predetermined threshold to the calculated first similarity. ### Claim 8 (depends on 5) The electronic device of claim 5, wherein the processor generates a set of pixels that neighbor a predetermined pixel that enters the mask head as an input, calculates third similarity between the predetermined pixel and neighboring pixels within the generated set of pixels, and matches the results of object recognition and the boundary portion based on whether the calculated third similarity is greater than a predetermined threshold.