Multi-image search
AssigneeGoogle Inc.
InventorsBelinda Luna Zeng, Harshit Kharbanda, Christopher James Kelley, Erica Bjornsson, David William Hendon
Life of this patent
Estimating the term…
Abstract
Systems and methods for multi-image search can include obtaining two or more images and determining one or more search results that are based on the two or more images. The one or more search results can be determined based on determined shared attributes of the two or more images. The one or more search results may be based on feature embeddings associated with the two or more images. The two or more images may be obtained based on one or more user interactions with one or more databases.
Claims20
A computer-implemented method for searching with multiple images, the method comprising:
- obtaining, by a computing system comprising one or more processors, a first image and a second image;
- processing, by the computing system, the first image with an embedding model to generate a first embedding, wherein the first embedding is associated with an embedding space;
- processing, by the computing system, the second image with the embedding model to generate a second embedding, wherein the second embedding is associated with the embedding space;
- determining, by the computing system, a third embedding based on the first embedding, the second embedding, and one or more learned distributions of the embedding space;
- determining, by the computing system, one or more search results based on the third embedding, wherein the one or more search results are associated with one or more search result embeddings determined to be associated with the third embedding;
- and
- providing, by the computing system, the one or more search results for display in a search results interface.
A computing system for searching with multiple images, the system comprising:
- one or more processors;
- and
- one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
- obtaining a search query, wherein the search query comprises a first image and a second image;
- processing the first image and the second image to determine one or more shared attributes, wherein the first image is associated with the one or more shared attributes, and wherein the second image is associated with the one or more shared attributes;
- processing the first image with an embedding model to generate a first embedding, wherein the first embedding is associated with an embedding space;
- processing the second image with the embedding model to generate a second embedding, wherein the second embedding is associated with the embedding space;
- determining the first embedding and the second embedding are associated with one or more learned distributions of the embedding space;
- determining one or more search results based at least in part on the first embedding, the second embedding, and the one or more learned distributions, wherein the one or more search results are determined to be associated with the one or more learned distributions;
- and
- providing the one or more search results in a search results interface.
One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising:
- obtaining a first image and a second image;
- processing the first image with an embedding model to generate a first embedding, wherein the first embedding is associated with an embedding space;
- processing the second image with the embedding model to generate a second embedding, wherein the second embedding is associated with the embedding space;
- determining a third embedding based on the first embedding, the second embedding, and one or more learned distributions of the embedding space;
- determining one or more search results based on the third embedding, wherein the one or more search results are associated with one or more search result embeddings determined to be associated with the third embedding;
- and
- providing the one or more search results for display in a search results interface.
Dependencies are inferred from the claim text.