# US 12,602,906 B2 · Image recognition apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,906 B2 - Document ID: US12602906B2 - Status: Granted - Kind: B2 - Application number: 18/160,186 - Filed: 2023-01-26 - Published: 2023-08-03 - Granted: 2026-04-14 - Earliest claimed: 2022-01-31 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Takamasa Tsunoda - Web page: https://patentgrep.com/patent/US12602906B2 ## CPC - G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods} - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking - G06V 10/82: using neural networks - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 20/53: {Recognition of crowd images, e.g. recognition of crowd congestion} - G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition} - G06V 2201/07: Target detection - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An image recognition apparatus that recognizes a target with respect to image data by detecting a plurality of targets with respect to image data and outputting a plurality of detection objects that is based on the detected plurality of targets, extracting respective feature quantities from the output plurality of detection objects, outputting, with respect to each of the detection objects, a filtered feature, which is a feature quantity obtained by filtering the feature quantity extracted from each of the detection objects, based on a first mask for current time for each detection object predicted at previous time, and predicting the first mask for next time for each of the detection objects. ## Claims ### Claim 1 An image recognition apparatus that recognizes a target with respect to image data, the image recognition apparatus comprising: a processor; and a memory storing executable instructions which, when executed by the processor, cause the image recognition apparatus to perform operations including: detecting a plurality of targets with respect to image data and outputting a plurality of detection objects that is based on the detected plurality of targets; extracting respective feature quantities from the output plurality of detection objects; outputting, with respect to each of the detection objects, a filtered feature, which is a feature quantity obtained by filtering the feature quantity extracted from each of the detection objects, based on a first mask for current time for each detection object predicted at previous time; predicting the first mask for next time for each of the detection objects; wherein each of the detection objects is a circumscribed rectangle of each of the targets; and wherein the first mask is a mask for a region in which each of the targets exists within the circumscribed rectangle. ### Claim 2 (depends on 1) The image recognition apparatus according to claim 1, wherein the operations further include assigning respective identifiers to the plurality of detection objects based on the filtered feature for each of the detection objects. ### Claim 3 (depends on 2) The image recognition apparatus according to claim 2, wherein a gathering of detection objects to which an identical identifier has been assigned at each of times is a track, wherein the operations further include calculating a cost for each track and each detection object based on a degree of similarity between filtered features for previous time of a plurality of tracks and a filtered feature for current time of each of the detection objects, and wherein the assigning includes assigning respective identifiers to the plurality of detection objects based on the cost. ### Claim 4 (depends on 3) The image recognition apparatus according to claim 3, wherein the operations further include updating track information by storing the output filtered feature for current time, the predicted first mask for next time, and the assigned identifier while associating them with each other with respect to each track. ### Claim 5 (depends on 3) The image recognition apparatus according to claim 3, wherein the operations further include, in a case where a degree of overlapping of a plurality of targets in each of the detection objects is less than a threshold value, outputting an average feature obtained by spatially averaging the extracted feature quantity, wherein the outputting includes, in a case where a degree of overlapping of a plurality of targets in each of the detection objects is greater than or equal to the threshold value, outputting the filtered feature, and wherein the calculating includes, in a case where a degree of overlapping of a plurality of targets in each of the detection objects is less than the threshold value, calculating a cost for each track and each detection object based on a degree of similarity between an average feature for previous time of a plurality of tracks and an average feature for current time in each of the detection objects, and, in a case where a degree of overlapping of a plurality of targets in each of the detection objects is greater than or equal to the threshold value, calculating a cost for each track and each detection object based on a degree of similarity between a filtered feature for previous time of a plurality of tracks and a filtered feature for current time in each of the detection objects. ### Claim 6 (depends on 5) The image recognition apparatus according to claim 5, wherein the operations further include, in a case where a degree of overlapping of a plurality of targets in each of the detection objects is greater than or equal to the threshold value, updating track information by storing the output filtered feature for current time, the predicted first mask for next time, and the assigned identifier while associating them with each other with respect to each track, and wherein the updating includes, in a case where a degree of overlapping of a plurality of targets in each of the detection objects is less than the threshold value, acquiring a filtered feature, which is a feature quantity obtained by filtering the extracted feature quantity, and updating track information by storing the acquired filtered feature for current time, the output average feature for current time, the predicted first mask for next time, and the assigned identifier while associating them with each other with respect to each track. ### Claim 7 (depends on 1) The image recognition apparatus according to claim 1, wherein the predicting includes predicting the first mask for next time for each of the detection objects based on the extracted feature quantity and a state of the predicting. ### Claim 8 (depends on 7) The image recognition apparatus according to claim 7, wherein the extracting includes extracting each of a first feature quantity and a second feature quantity from the plurality of detection objects, wherein the predicting includes predicting the first mask for next time for each of the detection objects based on the extracted first feature quantity and a state of the predicting, and wherein the outputting includes outputting, with respect to each of the detection objects, a filtered feature, which is a feature quantity obtained by filtering the extracted second feature quantity, based on the first mask for current time for each of the detection objects. ### Claim 9 (depends on 7) The image recognition apparatus according to claim 7, wherein the state of the predicting is a state of a first neural network. ### Claim 10 (depends on 9) The image recognition apparatus according to claim 9, wherein the first neural network is a recurrent neural network (RNN). ### Claim 11 (depends on 1) The image recognition apparatus according to claim 1, wherein the extracting and the outputting use a second neural network. ### Claim 12 (depends on 11) The image recognition apparatus according to claim 11, wherein the second neural network is a convolutional neural network (CNN). ### Claim 13 (depends on 1) The image recognition apparatus according to claim 1, wherein the predicting includes predicting the first mask and a second mask which is used for a region important for discriminating each of the targets, and wherein the outputting includes outputting, with respect to each of the detection objects, a filtered feature, which is a feature quantity obtained by filtering the extracted feature quantity, based on the first mask and the second mask for current time for each of the detection objects predicted at previous time. ### Claim 14 (depends on 13) The image recognition apparatus according to claim 13, wherein the outputting includes outputting the filtered feature based on a mask obtained by integrating the first mask and the second mask. ### Claim 15 (depends on 13) The image recognition apparatus according to claim 13, wherein the operations further include learning parameters for prediction of the second mask based on the extracted feature quantity. ### Claim 16 A recognition method for an image recognition apparatus that recognizes a target with respect to image data, the recognition method comprising: detecting a plurality of targets with respect to image data and outputting a plurality of detection objects that is based on the detected plurality of targets; extracting respective feature quantities from the output plurality of detection objects; outputting, with respect to each of the detection objects, a filtered feature, which is a feature quantity obtained by filtering the feature quantity extracted from each of the detection objects, based on a first mask for current time for each detection object predicted at previous time; predicting the first mask for next time for each of the detection objects; wherein each of the detection objects is a circumscribed rectangle of each of the targets; and wherein the first mask is a mask for a region in which each of the targets exists within the circumscribed rectangle. ### Claim 17 A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a computer, cause the computer to perform a recognition method for an image recognition apparatus that recognizes a target with respect to image data, the recognition method comprising: detecting a plurality of targets with respect to image data and outputting a plurality of detection objects that is based on the detected plurality of targets; extracting respective feature quantities from the output plurality of detection objects; outputting, with respect to each of the detection objects, a filtered feature, which is a feature quantity obtained by filtering the feature quantity extracted from each of the detection objects, based on a first mask for current time for each detection object predicted at previous time; predicting the first mask for next time for each of the detection objects; wherein each of the detection objects is a circumscribed rectangle of each of the targets; and wherein the first mask is a mask for a region in which each of the targets exists within the circumscribed rectangle.