Automatic suggestion of most informative images

A computer system, comprising:

• one or more processors;
• and
• a memory coupled to the one or more processors comprising instructions executable by the one or more processors, the one or more processors operable when executing the instructions to:
• using a server computer, obtain from a client computer a text input comprising one or more first unigrams in a query from a user;
• access in digital data storage coupled to the server computer a plurality of digital images, each of the plurality of digital images comprising one or more definition unigrams;
• train a deep learning model to map the one or more first unigrams to first vector representations for the text input and to map the one or more definition unigrams to second vector representations for the plurality of digital images, the deep learning model being a dual encoder model comprising a text encoder and an image encoder based on a ranking loss function;
• determine, using the deep learning model, the first vector representations of the text input by mapping the one or more first unigrams of the text input to the first vector representations for the text input;
• determine, using the deep learning model, a first embedding of the first vector representations of the text input in a multi-dimensional embedding space based on a combination of the first vector representations of the text input;
• determine, using the deep learning model, the second vector representations of each of the plurality of digital images by mapping the one or more definition unigrams of each of the plurality of digital images to the second vector representations for the plurality of digital images;
• determine, using the deep learning model, a second embedding of the second vector representations of each of the plurality of digital images in the multi-dimensional embedding space based on a combination of the second vector representations of a corresponding image;
• identify one or more relevant images based on a respective similarity of the first embedding to the second embedding;
• determine one or more information terms for each of the one or more relevant images, an image informativeness value for each of the one or more relevant images based on the one or more information terms, and a confidence score for each of the one or more information terms;
• and
• transmit, to the client computer in response to obtaining the text input, instructions for presenting a user interface comprising the one or more relevant images and the confidence score for each of the one or more information terms for each of the one or more relevant images.

One or more non-transitory computer-readable storage media storing one or more sequences of instructions which, when executed using one or more processors, cause the one or more processors to execute:

• using a server computer, obtain from a client computer a text input comprising one or more first unigrams in a query from a user;
• access in digital data storage coupled to the server computer a plurality of digital images, each of the plurality of digital images comprising one or more definition unigrams;
• train a deep learning model to map the one or more first unigrams to first vector representations for the text input and to map the one or more definition unigrams to second vector representations for the plurality of digital images, the deep learning model being a dual encoder model comprising a text encoder and an image encoder based on a ranking loss function;
• determine, using the deep learning model, the first vector representations of the text input by mapping the one or more first unigrams of the text input to the first vector representations for the text input;
• determine, using the deep learning model, a first embedding of the first vector representations of the text input in a multi-dimensional embedding space based on a combination of the first vector representations of the text input;
• determine, using the deep learning model, the second vector representations of each of the plurality of digital images by mapping the one or more definition unigrams of each of the plurality of digital images to the second vector representations for the plurality of digital images;
• determine, using the deep learning model, a second embedding of the second vector representations of each of the plurality of digital images in the multi-dimensional embedding space based on a combination of the second vector representations of a corresponding image;
• identify one or more relevant images based on a respective similarity of the first embedding to the second embedding;
• determine one or more information terms for each of the one or more relevant images, an image informativeness value for each of the one or more relevant images based on the one or more information terms, and a confidence score for each of the one or more information terms;
• and
• transmit, to the client computer in response to obtaining the text input, instructions for presenting a user interface comprising the one or more relevant images and the confidence score for each of the one or more information terms for each of the one or more relevant images.