Embedding contextual information in an image to assist understanding

A computer program product for embedding contextual information in an image or video frames, the computer program product comprising one or more computer readable storage mediums having program code embodied therewith, the program code comprising programming instructions for:

• training a generative adversarial network to provide contextual information to be embedded in said image or said video frames, wherein said contextual information comprises text, sound and/or video frames that provides context to said image or said video frames;
• receiving said image or said video frames;
• extracting features from said received image or said received video frames, wherein said extracted features comprise topics, events as well as sites and objects;
• identifying one or more images and/or one or more video frames in a database using said generative adversarial network associated with features with a similarity to said extracted features from said received image or said received video frames that exceeds a threshold value;
• extracting references containing contextual information associated with said identified one or more images and/or one or more video frames from said database using said generative adversarial network, wherein said references contain said contextual information to assist a user in understanding a meaning of said received image or said received video frames;
• and
• augmenting said image or said video frames with said extracted references.

A system, comprising:

• a memory for storing a computer program for embedding contextual information in an image or video frames;
• and
• a processor connected to said memory, wherein said processor is configured to execute program instructions of the computer program comprising:
• training a generative adversarial network to provide contextual information to be embedded in said image or said video frames, wherein said contextual information comprises text, sound and/or video frames that provides context to said image or said video frames;
• receiving said image or said video frames;
• extracting features from said received image or said received video frames, wherein said extracted features comprise topics, events as well as sites and objects;
• identifying one or more images and/or one or more video frames in a database using said generative adversarial network associated with features with a similarity to said extracted features from said received image or said received video frames that exceeds a threshold value;
• extracting references containing contextual information associated with said identified one or more images and/or one or more video frames from said database using said generative adversarial network, wherein said references contain said contextual information to assist a user in understanding a meaning of said received image or said received video frames;
• and
• augmenting said image or said video frames with said extracted references.