# US 2025/0095161 A1 · Track aware detection for object tracking systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095161 A1
- Document ID: US20250095161A1
- Status: Published application
- Kind: A1
- Application number: 18/400,323
- Filed: 2023-12-29
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Shay AMRAM; Zvi FIGOV; Moti KADOSH; Yonit HOFFMAN
- Web page: https://patentgrep.com/patent/US20250095161A1

## CPC

- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/56: relating to colour
- G06V 10/764: using classification, e.g. of video objects
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30241: Trajectory
- G06V 2201/07: Target detection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Examples of the present disclosure describe systems and methods for track aware object detection. In examples, image content comprising one or more objects is received. Frames in the image content are identified. Candidate bounding boxes are created around objects to be tracked in the frames and a confidence score is assigned to each candidate bounding box. The candidate bounding boxes for each object are compared to a predicted bounding box that is generated based on a current track for the object. Candidate bounding boxes that are determined to be similar to the predicted bounding box and/or that exceed a confidence score threshold are selected. The selected candidate bounding boxes are filtered until a single candidate bounding box that is most representative of each object to be tracked remains. The frame comprising the representative bounding box for each object is then added to a current track for the object.

## Claims

### Claim 1

A system comprising:
a processing system; and
memory comprising computer executable instructions that, when executed, perform operations comprising:
detecting an object in a current frame of image content, wherein detecting the object comprises creating multiple candidate bounding boxes for the object, each candidate bounding box comprising at least a portion of the object;
comparing the multiple candidate bounding boxes to a predicted bounding box for the object, wherein the predicted bounding box is generated based on a current track for the object, the current track including a first previous frame comprising the object;
based on the comparing, filtering the multiple candidate bounding boxes, wherein the filtering identifies a representative bounding box from the multiple candidate bounding boxes, the representative bounding box being a closest match to the predicted bounding box; and
adding the current frame comprising the representative bounding box to the current track.

### Claim 2 (depends on 1)

The system of claim 1, the operations further comprising:
prior to detecting the object in the current frame, receiving the image content at an image tracking system trained to detecting objects of interest and tracking movement of the objects of interest over a time period, wherein the image tracking system is stored in the memory of the system.

### Claim 3 (depends on 2)

The system of claim 2, wherein:
the objects of interest are predefined to include specified classes of objects; and
an object detector of the image tracking system is trained to detect the specified classes of objects.

### Claim 4 (depends on 2)

The system of claim 2, the operations further comprising:
in response to receiving the image content, using an image processor of the image tracking system to identify a plurality of frames in the image content, the plurality of frames including the current frame and the first previous frame; and
preprocessing a frame in the plurality of frames, the preprocessing including at least one of:
modifying color formatting of the frame;
modifying an aspect ratio of the frame; or
performing filtering or segmentation of the frame.

### Claim 5 (depends on 1)

The system of claim 1, wherein detecting the object further comprises assigning a confidence score to each candidate bounding box of the multiple candidate bounding boxes.

### Claim 6 (depends on 5)

The system of claim 5, wherein the confidence score represents a probability that the object belongs to a particular object class.

### Claim 7 (depends on 1)

The system of claim 1, wherein generating the predicted bounding box comprises determining a predicted location of the object in the current frame based on a first location of the object in the first previous frame.

### Claim 8 (depends on 7)

The system of claim 7, wherein:
the current track further includes a second previous frame comprising the object, the second previous frame being sequentially prior to the first previous frame; and
determining the predicted location of the object in the current frame comprises determining a motion path of the object based on the first location of the object in the first previous frame and a second location of the object in the second previous frame.

### Claim 9 (depends on 1)

The system of claim 1, wherein comparing the multiple candidate bounding boxes to the predicted bounding box comprises determining an Intersection over Union (IoU) score for each of the multiple candidate bounding boxes, the IoU score indicating an amount of similarity between the predicted bounding box and a respective candidate bounding box.

### Claim 10 (depends on 1)

The system of claim 1, wherein filtering the multiple candidate bounding boxes using a filtering algorithm to remove bounding boxes from the multiple candidate bounding boxes until one bounding box for the object remains, the one bounding box being the representative bounding box.

### Claim 11 (depends on 10)

The system of claim 10, wherein the filtering algorithm is a Non-Maximum Suppression (NMS) algorithm.

### Claim 12 (depends on 1)

The system of claim 1, wherein the current track represents a motion or a position of the object through multiple frames of the image content.

### Claim 13 (depends on 1)

The system of claim 1, wherein adding the current frame comprising the representative bounding box to the current track includes at least one of:
adding a confidence score for the representative bounding box to the current track; or
adding metadata for the representative bounding box to the current track.

### Claim 14

A method comprising:
receiving a vector embedding of a bounding box for an object in image content;
performing a clustering operation for the vector embedding based on at least one of:
a specified interval of frames of the image content have been evaluated; or
a specified number of bounding boxes for the object have been created,
the clustering operation including adding the vector embedding to a cluster of vector embeddings for the object based on a comparison of the vector embedding to an average vector embedding for the cluster of vector embeddings, wherein adding the vector embedding to the cluster of vector embeddings for the object comprises comparing the vector embedding to a plurality of clusters of vector embeddings for the object, each of the plurality of clusters of vector embeddings representing a different perspective of the object.

### Claim 15 (depends on 14)

The method of claim 14, wherein each cluster of the plurality of clusters of vector embeddings comprises:
an identifier for the object; and
an indication of a perspective represented by the cluster.

### Claim 16 (depends on 15)

The method of claim 15, wherein adding the vector embedding to the cluster of vector embeddings for the object further comprises:
determining a respective average vector embedding for each of the plurality of clusters of vector embeddings; and
comparing the vector embedding to each respective average vector embedding based on Euclidean distance or Cosine similarity.

### Claim 17

A method comprising:
detecting a completed track for an object being tracked in image content, the completed track comprising frames depicting the object over a first time period;
creating multiple sliding windows for the completed track, the multiple sliding windows each comprising a subset of the frames;
determining a specified number of the multiple sliding windows exceed a confidence threshold value based on a median confidence calculation for each of the multiple sliding window; and
based on the determining, persisting the completed track for a second time period.

### Claim 18 (depends on 17)

The method of claim 17, wherein the completed track represents a track in which the object is no longer appearing in current frames of the image content.

### Claim 19 (depends on 17)

The method of claim 17, wherein each frame of the frames is associated with a confidence score for the object, the confidence score representing a probability that the object belongs to a particular object class.

### Claim 20 (depends on 17)

The method of claim 17, wherein the median confidence calculation for each of the multiple sliding windows is based on a median value of confidence scores associated with frames in the multiple sliding windows.
