US 2022/0092105 A1 · Intelligent Systems and Methods for Visual Search Queries

Abstract

A user can submit a visual query that includes one or more images. Various processing techniques such as optical character recognition (OCR) techniques can be used to recognize text (e.g. in the image, surrounding image(s), etc.) and/or various object detection techniques (e.g., machine-learned object detection models, etc.) may be used to detect objects (e.g., products, landmarks, animals, humans, etc.) within or related to the visual query. Content related to the detected text or object(s) can be identified and potentially provided to a user as search results or a proactive content feed. As such, aspects of the present disclosure enable the visual search system to more intelligently process a visual query to provide improved search results and content feeds, including those search results which are more personalized and/or consider contextual signals to account for implicit characteristics of the visual query and/or user's search intent.

Claims38

Dependent claims

A computer-implemented method for providing personalized visual search query result notifications within a user interface overlaid upon imagery, the method comprising:

obtaining, by a computing system comprising one or more computing devices, a visual search query associated with a user, wherein the visual search query comprises an image;
identifying, by the computing system, a plurality of candidate search results for the visual search query, wherein each candidate search result is associated with a particular sub-portion of the image, and wherein a plurality of candidate visual result notifications are respectively associated with the plurality of candidate search results;
accessing, by the computing system, user-specific user interest data associated with the user and descriptive of visual interests of the user;
generating, by the computing system, a ranking of the plurality of candidate search results based at least in part on a comparison of the plurality of candidate search results to the user-specific user interest data associated with the user;
selecting, by the computing system, at least one of the plurality of candidate search results as at least one selected search result based at least in part on the ranking;
and
providing, by the computing system, at least one selected visual result notification respectively associated with the at least one selected search result for overlay upon the particular sub-portion of the image associated with the selected search result.

4 more elements

A computing system that returns content for multiple canonical items responsive to visual search queries, the computing system comprising:

one or more processors;
and
one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining a visual search query, wherein the visual search query comprises an image that depicts an object;
accessing a graph that describes a plurality of different items, wherein a respective set of content is associated with each of the plurality of different items;
selecting, based on the visual search query, a plurality of selected items from the graph for the object depicted by the image;
in response to the visual search query, returning a combined set of content as search results, wherein the combined set of content comprises at least a portion of the respective set of content associated with each of the plurality of selected items.

4 more elements

A computing system to disambiguate between object-specific and categorical visual queries, the computing system comprising:

one or more processors;
and
one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining a visual search query, wherein the visual search query comprises an image that depicts one or more objects;
identifying one or more compositional characteristics of the image included in the visual search query;
determining, based at least in part on the one or more compositional characteristics of the image included in the visual search query, whether the visual search query comprises an object-specific query that pertains specifically to the one or more objects identified within the image included in the visual search query or whether the visual search query comprises a categorical query that pertains to a general category of the one or more objects identified within the image included in the visual search query;
when it is determined that the visual search query comprises an object-specific query, returning one or more object-specific search results that pertain specifically to the one or more objects identified within the image included in the visual search query;
and
when it is determined that the visual search query comprises a categorical query, returning one or more categorical search results that pertain to the general category of the one or more objects identified within the image included in the visual search query.

6 more elements

Dependencies are inferred from the claim text.

Intelligent Systems and Methods for Visual Search Queries

Life of this patent

Intelligent Systems and Methods for Visual Search Queries

Life of this patent

Abstract

Claims38