US 10,949,708 B2 · Fine-grained image similarity

Abstract

Methods, systems, and apparatus, for determining fine-grained image similarity. In one aspect, a method includes training an image embedding function on image triplets by selecting image triplets of first, second and third images; generating, by the image embedding function, a first, second and third representations of the features of the first, second and third images; determining, based on the first representation of features and the second representation of features, a first similarity measure for the first image to the second image; determining, based on the first representation of features and the third representation of features, a second similarity measure for the the first image to the third image; determining, based on the first and second similarity measures, a performance measure of the image embedding function for the image triplet; and adjusting the parameter weights of the image embedding function based on the performance measures for the image triplets.

Claims20

Dependent claims

A computer-implemented method, comprising:

receiving, at a search system, a search query comprising a search query image;
determining one or more similar images that are similar to the search query image using a trained model function of the search system, wherein the trained model function comprises an image embedding function that operates on the search query image to produce a representation of features of the search query image, wherein the image embedding function is trained using one or more image triplets selected based on pairwise relevance scores, each image triplet of the one or more image triplets being a combination of a first image, a second image and a third image;
and
providing the one or more similar images using the search system.

1 more element

A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that, when executed by a search system, cause the search system to perform operations comprising:

receiving a search query comprising a search query image;
determining one or more similar images that are similar to the search query image using a trained model function of the search system, wherein the trained model function comprises an image embedding function that operates on the search query image to produce a representation of features of the search query image, wherein the image embedding function is trained using one or more image triplets selected based on pairwise relevance scores, each image triplet of the one or more image triplets being a combination of a first image, a second image and a third image;
and
providing the one or more similar images.

1 more element

Dependencies are inferred from the claim text.

Fine-grained image similarity

Life of this patent

Fine-grained image similarity

Life of this patent

Abstract

Claims20