Relevance-based image selection

One or more non-transitory computer readable storage media storing computer executable code that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

• storing a searchable video index associated with a plurality of videos that each include a plurality of frames, the searchable video index including a mapping between two or more frames of each of the plurality of videos and one or more keyword representations, the mapping generated based at least in part on a machine-learned model trained to learn correlations between visual content of individual video frames and keyword representations;
• receiving a search query including one or more textual keywords;
• selecting a first video from the searchable video index based on the one or more textual keywords;
• determining a frame of the first video based at least in part on the one or more textual keywords and the mapping between the two or more frames of each of the plurality of videos and the one or more keyword representations;
• and
• providing a result set including the frame of the first video and a link to the first video in response to the search query.

A computing system having one or more processors configured to perform operations comprising:

• receiving a labeled training dataset comprising a set of training videos together with textual data descriptive of visual content for each of two or more frames of each training video;
• extracting from the set of training videos one or more features characterizing the visual content of each of the two or more frames from each of the training videos;
• training a machine-learned model based at least in part on the labeled training dataset to learn correlations between the one or more features characterizing the visual content of each of the two or more frames and one or more keyword representations;
• inputting to the machine-learned model data associated with two or more frames of each of a plurality of videos associated with a searchable video index;
• generating, for each of the two or more frames of the plurality of videos associated with the searchable video index, at least one keyword representation based on one or more outputs of the machine-learned model for said each of the two or more frames;
• and
• storing the searchable video index including a mapping between the two or more frames of each of the plurality of videos and the at least one keyword representation.

The computing system of 15, wherein:

• the one or more features are one or more feature vectors;
• the operations further comprise extracting a plurality of keyword representations from the labeled training dataset;
• and
• training the machine-learned model includes training the machine-learned model to learn correlations between the one or more feature vectors and the one or more keyword representations based at least in part of the plurality of keyword representations extracted from the labeled training dataset.