Back to results
US 12,524,459 B1Granted
Artificial intelligence-based image search refinement
AssigneeDeepMind Technologies Limited
InventorAditya Avinash
Life of this patent
Estimating the term…
Abstract
Systems and methods for image search result filtering can include obtaining a search query, determining a plurality of candidate image search results, processing the search query with a generative model to determine a plurality of search result criteria, and refining the plurality of candidate image search results based on determining whether the candidate results satisfy the plurality of search results criteria. The systems and methods can perform a plurality of determinations based on the output of the generative model.
Claims19
A computing system for image search, the system comprising:
- one or more processors;
- and
- one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
- obtaining a search query, wherein the search query comprises one or more inputs requesting image data comprising one or more particular feature sets;
- determining a plurality of candidate image search results based on the search query;
- processing the search query with a generative model to generate a plurality of search result criteria, wherein the generative model comprises a router model trained to interact with external tools and determine a next task in performing search query response processing;
- processing, for at least a subset of the plurality of candidate image search results and based on a tool call generated with the router model of the generative model, a respective candidate image search result of the plurality of candidate image search results to determine a plurality of scores associated with the plurality of search result criteria;
- determining a plurality of respective search result rankings of the plurality of candidate image search results based on each of the plurality of scores;
- and
- in response to determining the plurality of respective search result rankings of the plurality of candidate image search results based on each of the plurality of scores, providing a refined image search results list for display.
7 more elements
A computer-implemented method for image search, the method comprising:
- obtaining, by a computing system comprising one or more processors, a search query, wherein the search query comprises one or more inputs requesting image data comprising one or more particular objects and one or more particular actions;
- determining, by the computing system and via an embedding-based search performed based on generating a query embedding with an embedding model in response to processing the search query, a plurality of candidate image search results and a plurality of respective search result rankings based on the search query;
- processing, by the computing system, the search query with a generative model to generate a plurality of search result criteria, wherein the generative model comprises an autoregressive language model, wherein the plurality of search result criteria is descriptive of the one or more particular objects and the one or more particular actions;
- for at least a subset of the plurality of candidate image search results:
- processing, by the computing system and based on the plurality of search result criteria and based on a tool call generated with a router model of the generative model, a respective candidate image search result of the plurality of candidate image search results with one or more machine-learned classification models to perform a plurality of classifications associated with the plurality of search result criteria;
- adjusting, by the computing system and with an auto-rater configured for low-latency rating, the plurality of respective search result rankings of the plurality of candidate image search results based on each of the plurality of classifications;
- and
- in response to adjusting the plurality of respective search result rankings of the plurality of candidate image search results based on each of the plurality of classifications, providing, by the computing system, a refined image search results list for display, wherein the refined image search results list comprises a set of image search results that comprise classifications descriptive of the plurality of search result criteria being met.
5 more elements
One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising:
- obtaining a search query, wherein the search query comprises one or more inputs requesting image data comprising a plurality of particular feature sets;
- determining, via an embedding-based search performed based on generating a query embedding with an embedding model in response to processing the search query, a plurality of candidate image search results and a plurality of respective search result rankings based on the search query;
- processing the search query, the plurality of candidate image search results, and the plurality of respective search result rankings with a generative model to determine a refined set of image search results, wherein the generative model comprises an autoregressive language model, wherein the generative model comprises an artificial intelligence agent tuned to determine a next task in a data processing instance and interact with data processing tools, wherein processing the search query, the plurality of candidate image search results, and the plurality of respective search result rankings with a generative model to determine the refined set of image search results comprises:
- processing the search query with a generative model to generate a routing output comprising a plurality of search result criteria and a plurality of routing calls for interfacing with a plurality of different classification models associated with different criteria of the plurality of search result criteria;
- interfacing, based on the plurality of routing calls generated with a router model of the generative model, with the plurality of different classification models to perform a plurality of classifications for each of the plurality of candidate image search results;
- processing the plurality of candidate image search results, the plurality of search result criteria, and each of the plurality of classifications to generate the refined set of image search results descriptive of a subset of the plurality of candidate image search results that comprise the plurality of particular feature sets;
- and
- providing the refined set of image search results for display in a graphical search results interface.
5 more elements
Dependencies are inferred from the claim text.