Back to results
US 10,762,644 B1Granted
Multiple object tracking in video by combining neural networks within a bayesian framework
AssigneeAmazon Technologies, Inc.
InventorsVijay Mahadevan, Stefano Soatto
Life of this patent
Estimating the term…
Abstract
Techniques for multiple object tracking in video are described in which the outputs of neural networks are combined within a Bayesian framework. A motion model is applied to a probability distribution representing the estimated current state of a target object being tracked to predict the state of the target object in the next frame. A state of an object can include one or more features, such as the location of the object in the frame, a velocity and/or acceleration of the object across frames, a classification of the object, etc. The prediction of the state of the target object in the next frame is adjusted by a score based on the combined outputs of neural networks that process the next frame.
Claims20
A computer-implemented method comprising:
- generating, based at least in part on a current state probability distribution of a target object of a plurality of target objects in a current frame of a video, a future state probability distribution of the target object in a next frame of the video, wherein a future state in the future state probability distribution includes a possible location of the target object in the next frame and a current state in the current state probability distribution includes a possible location of the target object in the current frame, and wherein the possible location of the target object in the next frame is generated with a motion model applied to the possible location of the target object in the current frame;
- generating, from the next frame, a state likelihood model of the target object in the next frame from a classification of a region in the next frame and a comparison of a template of the target object from the current frame with the next frame, wherein the region and the template are based at least in part on a portion of the current frame that includes the possible location of the target object in the current frame;
- determining a current state probability distribution of the target object in the next frame by applying the state likelihood model to the future state probability distribution of the target object in the current frame;
- and
- causing a representation of an estimated location of the target object in the next frame to be displayed based at least in part on a component of the current state probability distribution of the target object in the next frame.
2 more elements
A computer-implemented method comprising:
- generating, at least in part with a motion model, a future state probability distribution of a target object in a next frame of a video from a current state probability distribution of the target object in a current frame of the video;
- generating, from the next frame, one or more values to adjust the future state probability distribution of the target object, the one or more values based at least in part on a classification of a region in the next frame and a comparison of a template of the target object from the current frame with the next frame, wherein the region and the template are based at least in part on a portion of the current frame that includes a possible location of the target object in the current frame from the current state probability distribution of the target object in the current frame;
- determining a current state probability distribution of the target object in the next frame by adjusting the future state probability distribution of the target object in the current frame by the one or more values;
- and
- causing a representation of an estimated location of the target object in the next frame to be displayed based at least in part on a component of the current state probability distribution of the target object in the next frame.
2 more elements
A system comprising:
- a storage device to store a current frame of a video and a next frame of the video;
- and
- a multiple object tracking service implemented by one or more electronic devices, the multiple object tracking service including instructions that upon execution cause the multiple object tracking service to:
- obtain the current frame and the next frame from the storage device;
- generate, at least in part with a motion model, a future state probability distribution of a target object in the next frame from a current state probability distribution of the target object in the current frame;
- generate, from the next frame, one or more values to adjust the future state probability distribution of the target object, the one or more values based at least in part on a classification of a region in the next frame and a comparison of a template of the target object from the current frame with the next frame, wherein the region and the template are based at least in part on a portion of the current frame that includes a possible location of the target object in the current frame from the current state probability distribution of the target object in the current frame;
- determine a current state probability distribution of the target object in the next frame by adjusting the future state probability distribution of the target object in the current frame by the one or more values;
- and
- cause a representation of an estimated location of the target object in the next frame to be displayed based at least in part on a component of the current state probability distribution of the target object in the next frame.
6 more elements
Dependencies are inferred from the claim text.