# US 10,769,808 B2 · Apparatus and methods of automated tracking and counting of objects on a resource-constrained device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,808 B2 - Document ID: US10769808B2 - Status: Granted - Kind: B2 - Application number: 15/901,397 - Filed: 2018-02-21 - Published: 2019-04-25 - Granted: 2020-09-08 - Earliest claimed: 2017-10-20 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Donna Katherine Long; Arthur Charles Tomlin; Kenneth Liam Kiemele; John Benjamin Hesketh - Web page: https://patentgrep.com/patent/US10769808B2 ## CPC - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/96: Management of image or video recognition tasks - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06T 2207/20081: Training; Learning - G06T 2207/30242: Counting objects in image - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The present disclosure provides apparatus and methods for automated tracking and counting of objects in a set of image frames using a resource-constrained device based on analysis of a selected subset of image frames, and based on selectively timing when resource-intensive operations are performed. ## Claims ### Claim 1 An automated object tracking and counting system, comprising: a memory comprising instructions; and a processor in communication with the memory and configured to execute the instructions, wherein the processor is configured to: obtain a set of image frames captured over time; detect one or more objects in the set of image frames; track positioning of the detected one or more objects in each of the set of image frames; determine whether a current timing condition meets one or more classification timing rules; select a subset of image frames from the set of image frames based on a selection parameter of each of the subset of image frames meeting a selection criteria, in response to the current timing condition meeting the one or more classification timing rules, wherein the selection parameter includes a distance of a position of each of the subset of image frames to a target position of a target image frame, and wherein the selection criteria comprises a number of image frames having a lowest distance to the target position of the target image frame; classify as a respective object type each of the detected one or more objects; count a number of the classified object types in the subset of image frames; and output the number of the classified object types as a current count of the classified object type. ### Claim 2 (depends on 1) The system of claim 1, wherein the selection parameter includes an amount that each of the subset of image frames covers a target image frame, and wherein the selection criteria comprises a number of image frames having a highest amount of coverage. ### Claim 3 (depends on 1) The system of claim 1, wherein the selection parameter includes a direction of movement of each of the subset of image frames, and wherein the selection criteria comprises a defined direction of movement. ### Claim 4 (depends on 1) The system of claim 1, wherein the one or more classification timing rules allow for adjustability in balancing classification quality and processing speed. ### Claim 5 (depends on 1) The system of claim 1, wherein the one or more classification timing rules include enabling the processor to perform one or more of the selecting, the classifying, or the counting: only when the detecting or the tracking is not operating; when a number of the subset of image frames or a memory size of the number of the subset of image frames are pending processing; or when at least one of the subset of image frames is pending processing and a time since a last one of the selecting, the classifying, or the counting was performed satisfies a threshold. ### Claim 6 (depends on 1) The system of claim 1, wherein the processor is further configured to add the current count corresponding to the number of the classified object types to a value of a previous total count of the classified object types to define a current total count, and to output the current total count. ### Claim 7 (depends on 1) The system of claim 1, wherein the processor is further configured to: obtain another set of image frames captured over time in response to the current timing condition not meeting the one or more classification timing rules; and perform the detecting and tracking for the another set of image frames. ### Claim 8 An automated method of counting objects, comprising: receiving a plurality of image frames from a camera; detecting at least a first unidentified object in each of a first set of the plurality of image frames; tracking at least the first unidentified object through the first set of the plurality of image frames; determining a selection parameter associated with each of the first set of the plurality of image frames; selecting a subset of image frames from the first set of the plurality of image frames based on each selection parameter of the subset of image frames meeting a selection criteria, wherein each selection parameter includes a distance of a position of each of the subset of image frames to a target position of a target image frame, and wherein the selection criteria comprises a number of image frames having a lowest distance to the target position of the target image frame; determining that a classification timing trigger meets a trigger condition; automatically classifying at least the first unidentified object as a first type of object based on analyzing the subset of image frames in response to the classification timing trigger meeting the trigger condition; identifying a number of the first type of object in the subset of image frames to define a current count; updating a total count of the first type of object based on the number of the first type of object defined by the current count; and outputting the current count and/or the total count of the first type of object. ### Claim 9 (depends on 8) The method of claim 8, wherein the selection parameter includes an amount that each of the subset of image frames covers a target image frame, and wherein the selection criteria comprises a number of image frames having a highest amount of coverage. ### Claim 10 (depends on 8) The method of claim 8, wherein the selection parameter includes a direction of movement of each of the subset of image frames, and wherein the selection criteria comprises a defined direction of movement. ### Claim 11 (depends on 8) The method of claim 8, wherein the trigger condition allows for adjustability in balancing classification quality and processing speed. ### Claim 12 (depends on 8) The method of claim 8, wherein the trigger condition includes enabling the method to perform one or more of the selecting, the classifying, or the counting: only when the detecting or the tracking is not operating; when a number of the subset of image frames or a memory size of the number of the subset of image frames are pending processing; or when at least one of the subset of image frames is pending processing and a time since a last one of the selecting, the classifying, or the counting was performed satisfies a threshold. ### Claim 13 (depends on 8) The method of claim 8, further comprising adding the current count corresponding to the number of the classified object types to a value of a previous total count of the classified object types to define a current total count, and to output the current total count. ### Claim 14 (depends on 8) The method of claim 8, further comprising: obtaining another set of image frames captured over time in response to the classification timing trigger not meeting the trigger condition; and performing the detecting and tracking for the another set of image frames. ### Claim 15 A non-transitory computer-readable medium storing instructions for automated object tracking and counting that are executable by a processor, comprising: instructions to cause the processor to obtain a set of image frames captured over time; instructions to cause the processor to detect one or more objects in the set of image frames; instructions to cause the processor to track positioning of the detected one or more objects in each of the set of image frames; instructions to cause the processor to determine whether a current timing condition meets one or more classification timing rules; instructions to cause the processor to select a subset of image frames from the set of image frames based on a selection parameter of each of the subset of image frames meeting a selection criteria, in response to the current timing condition meeting the one or more classification timing rules, wherein the selection parameter includes a distance of a position of each of the subset of image frames to a target position of a target image frame, and wherein the selection criteria comprises a number of image frames having a lowest distance to the target position of the target image frame; instructions to cause the processor to classify as a respective object type each of the detected one or more objects; instructions to cause the processor to count a number of the classified object types in the subset of image frames; and instructions to cause the processor to output the number of the classified object types as a current count of the classified object type. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the one or more classification timing rules allow for adjustability in balancing classification quality and processing speed. ### Claim 17 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the one or more classification timing rules include enabling the processor to perform one or more of the selecting, the classifying, or the counting: only when the detecting or the tracking is not operating; when a number of the subsets of image frames or a memory size of the number of the subsets of image frames are pending processing; or when at least one of the subset of image frames is pending processing and a time since a last one of the selecting, the classifying, or the counting was performed satisfies a threshold.