Contextual digital media processing systems and methods

A method, comprising:

• receiving a live content capture from a gaming console, the live content capture comprising images or video that are being displayed to a user on a display device communicatively coupled to the gaming console;
• processing the images or video in real time, the processing comprising:
• analyzing the live content capture to extract contextual information from the live content capture using a plurality of contextual information sources comprising any of optical character recognition to detect keywords or phrases, a reverse image search using a search engine for related images, one or more of a scene detection, a discrete feature detection, and a landmark detection, said detections using machine learning to recognize features in the images or video;
• timestamping a series of frames of the images or video;
• applying a weighting to results of each of the reverse image search and results of performed said detections, the weighting based on a predetermined value assigned to a type of contextual evaluation process;
• and
• generating a batch of contextual information over a defined timeframe;
• storing a most recent batch of contextual information processed for the live content capture in real time, the most recent batch of contextual information being stored in memory of a contextual repository, wherein the most recent batch of contextual information changes as the images or video change;
• performing a first assessment, the first assessment being of contextual information of the batch of contextual information for relevancy using an aggregate of confidence scores and commonality calculated across the plurality of contextual information sources, the aggregate of confidence scores comprising a first confidence score based on frequency of groups of textual data detected by optical character recognition, a second confidence score associated with images identified based on the reverse image search, and a third confidence score associated with one or more features recognized in the live content capture based on the machine learning;
• searching the Internet for a third-party supplementary content based using a search input, the search input being determined by the first assessment;
• transmitting, in real time, the third-party supplementary content to the gaming console;
• and
• generating, by the gaming console, an updated visual display, the updated visual display comprising the live content capture and an overlay, the overlay comprising the supplementary content.

A system, comprising:

• a processor;
• and
• a memory communicatively coupled to the processor, the memory storing instruction executable by the processor to perform a method comprising:
• receiving a digital media capture of live media as image or video content from a source device, the source device being communicatively coupled with a display medium, the source device operating to generate a visual display and a user interface on the display medium;
• analyzing the digital media capture to extract contextual information from the digital media capture by processing the image or video content using a plurality of contextual information sources comprising any of optical character recognition to detect keywords or phrases, a reverse image search using a search engine for related images, one or more of a scene detection, a discrete feature detection, and a landmark detection, said detections using machine learning to recognize features in the image or video content, timestamping a series of frames of the image or video content, applying a weighting to results of each of the reverse image search and results of performed said detections, the weighting based on a predetermined value assigned to a type of contextual evaluation process, and generating a batch of contextual information over a timeframe;
• performing an assessment, the assessment being of contextual information of the batch of contextual information for relevancy using an aggregate of confidence scores and commonality calculated across the plurality of contextual information sources, the aggregate of confidence scores comprising a first confidence score based on frequency of groups of textual data detected by optical character recognition, a second confidence score associated with images identified based on the reverse image search, and a third confidence score associated with one or more features recognized in the digital media capture based on the machine learning;
• searching at least one network for supplementary content using a search input, the search input being determined by the assessment;
• transmitting the supplementary content to the source device;
• generating, by the source device, an updated visual display, the updated visual display comprising the image or video content and an overlay, the overlay comprising the supplementary content;
• and
• storing the image or video content, the contextual information of the batch of contextual information, and the supplementary content.