Computer vision on broadcast video

A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, that when executed by a computing device, performs a method comprising:

• streaming, by the computing device over a network to a device of a user, a streaming media file comprising a plurality of frames of content;
• receiving, by the computing device over the network from the user device, an indication of a request for additional content, the indication identifying a set of frames of the streaming media file;
• analyzing, by the computing device, the identified set of frames, said analysis comprising identifying a set of content objects that are within the set of frames, and identifying information related to the set of frames and the set of content objects;
• creating, by the computing device, a queue based on the identified information related to the set of frames and the set of content objects, the queue comprising data for each of the frames in the set of frames and data indicating where and when each content object within the set of content objects appears within each frame;
• executing, by the computing device, a trained network model based on the queue, the execution being based on input comprising the information related to the set of frames and the set of content objects;
• identifying, by the computing device, based on the execution, a content object from the set of content objects;
• retrieving, by the computing device, augmenting information related to the content object;
• and
• communicating, by the computing device, said augmenting information to the user device for display in conjunction with the display of the content object.

A computing device comprising:

• a processor;
• and
• a non-transitory computer-readable storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:
• logic executed by the processor for streaming, over a network to a device of a user, a streaming media file comprising a plurality of frames of content;
• logic executed by the processor for receiving, over the network from the user device, an indication of a request for additional content, the indication identifying a set of frames of the streaming media file;
• logic executed by the processor for analyzing the identified set of frames, said analysis comprising identifying a set of content objects that are within the set of frames, and identifying information related to the set of frames and the set of content objects;
• logic executed by the processor for creating a queue based on the identified information related to the set of frames and the set of content objects, the queue comprising data for each of the frames in the set of frames and data indicating where and when each content object within the set of content objects appears within each frame;
• logic executed by the processor for executing a trained network model based on the queue, the execution being based on input comprising the information related to the set of frames and the set of content objects;
• logic executed by the processor for identifying, based on the execution, a content object from the set of content objects;
• logic executed by the processor for retrieving augmenting information related to the content object;
• and
• logic executed by the processor for communicating said augmenting information to the user device for display in conjunction with the display of the content object.