Adaptive object tracking policy

A system comprising:

• one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
• receiving a current video frame by a user device having a plurality of installed object trackers, wherein each object tracker is configured to perform a different object tracking procedure on the current video frame;
• obtaining, by the user device, a plurality of object tracks previously generated by one or more of the plurality of installed object trackers, wherein each object track specifies a different respective region of a previous video frame in which a respective object was detected;
• providing (i) the current video frame and (ii) the plurality of object tracks previously generated by one or more of the object trackers as input to a trained policy engine that implements a reinforcement learning model to generate a particular object tracking plan,
• wherein the reinforcement learning model is trained to use the current video frame and the plurality of object tracks to generate an object tracking plan based on a predicted measure of accuracy and a predicted measure of computational complexity of the object tracking plan, each object tracking plan comprising data representing a set of object trackers to be performed on the current video frame;
• receiving an output of the reinforcement learning model representing an object tracking plan having a plurality of different object trackers to be applied to different respective regions of the current video frame based on the plurality of object tracks;
• and
• performing, by the user device, the object tracking plan on the current video frame to generate one or more updated object tracks for the current video frame.

One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving a current video frame by a user device having a plurality of installed object trackers, wherein each object tracker is configured to perform a different object tracking procedure on the current video frame;
• obtaining, by the user device, a plurality of object tracks previously generated by one or more of the plurality of installed object trackers, wherein each object track specifies a different respective region of a previous video frame in which a respective object was detected;
• providing (i) the current video frame and (ii) the plurality of object tracks previously generated by one or more of the object trackers as input to a trained policy engine that implements a reinforcement learning model to generate a particular object tracking plan,
• wherein the reinforcement learning model is trained to use the current video frame and the plurality of object tracks to generate an object tracking plan based on a predicted measure of accuracy and a predicted measure of computational complexity of the object tracking plan, each object tracking plan comprising data representing a set of object trackers to be performed on the current video frame;
• receiving an output of the reinforcement learning model representing an object tracking plan having a plurality of different object trackers to be applied to different respective regions of the current video frame based on the plurality of object tracks;
• and
• performing, by the user device, the object tracking plan on the current video frame to generate one or more updated object tracks for the current video frame.