Interactive viewing experiences by detecting on-screen text

A method comprising:

• receiving, by a computer system, video data comprising a plurality of frames arranged in an order;
• for a frame in the plurality of frames:
• determining, by the computer system, contextual data associated with the video data;
• determining, by the computer system, a region containing text based on the contextual data;
• performing, by the computer system, a character recognition operation on the region to generate recognized characters;
• generating, by the computer system, textual data based on the recognized characters;
• and
• generating, by the computer system, a graphical user interface element definition corresponding to the region based on the textual data

A computing system comprising:

• one or more processors;
• and
• a memory comprising instructions that, when executed by the processors, configure the one or more processors to be configured to:
• receive video data comprising a plurality of frames arranged in an order;
• for a frame in the plurality of frames:
• determine a region containing text;
• perform a character recognition operation on the region to generate recognized characters;
• generate textual data based on the recognized characters;
• and
• generate a graphical user interface element definition corresponding to the region based on the textual data.