# US 8,751,530 B1 · Visual restrictions for image searches > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,751,530 B1 - Document ID: US8751530B1 - Status: Granted - Kind: B1 - Application number: 13/565,149 - Filed: 2012-08-02 - Granted: 2014-06-10 - Earliest claimed: 2012-08-02 - Assignees: Google Inc. - Applicants: Sergey Ioffe - Inventors: Sergey Ioffe - Web page: https://patentgrep.com/patent/US8751530B1 ## CPC - G06F 16/532: Query formulation, e.g. graphical querying - G06F 16/583: using metadata automatically derived from the content - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and apparatus are provided for performing an image search based on a search query having a portion P1 and a portion P2. Based on the first search query, a second search query is generated that includes a portion P3 and the portion P2, such that the second search query is broader in scope than the first search query, while still retaining the portion P2 of the first query. A first image search is then performed for the first search query to obtain a first set of search results and a second image search is performed for the second search query to obtain a second set of search results. Consequently, an image from the first set of search results is selected for presentation to a user, wherein the selection is based on content of the second set of search results. ## Claims ### Claim 1 A computer-implemented method comprising: receiving an initial search query that includes (i) one or more query terms associated with an object, and (ii) one or more query terms that identify a context in which the object is to appear in images that are identified in response to the initial search query; determining a category associated with the object; generating a generalized search query that includes (i) one or more query terms associated with the category, and (ii) one or query terms that identify the context in which the object is to appear in images that are identified in response to the initial search query; obtaining (i) an initial set of one or more images that are identified as responsive to the initial search query, and (ii) a generalized set of one or more images that are identified as responsive to the generalized search query; and selecting one or more of the images of the initial set based at least on an indication of similarity between the selected one or more images and the one or more images of the generalized set of one or more images. ### Claim 2 (depends on 1) The method of claim 1, wherein the context comprises a viewpoint. ### Claim 3 (depends on 1) The method of claim 1, wherein determining a category associated with the object comprises identifying categories associated with the one or more query terms associated with the object in the initial search query. ### Claim 4 (depends on 1) The method of claim 1, comprising generating the indication of similarity based on: determining a function based on the generalized set of one or more images and the first query; and determining to generate the indication of similarity based on evaluating the function using the selected one or more images. ### Claim 5 (depends on 1) The method of claim 1, comprising generating the indication of similarity based on: training an image classifier using the one or more images of the generalized set of one or more images; and determining to generate the indication of similarity based on classifying the selected one or more images using the trained image classifier. ### Claim 6 (depends on 1) The method of claim 1, comprising generating the indication of similarity based on: generating first representations of the selected one or more images; generating a second representation based on the generalized set of one or more images; identifying differences between the first representations and the second representation; and determining to generate the indication of similarity based on the identified differences. ### Claim 7 (depends on 1) The method of claim 1, comprising: identifying, in the initial search query, the one or more query terms associated with the object; and identifying, in the initial search query, the one or more query terms that identify the context in which the object is to appear in images that are identified in response to the initial search query. ### Claim 8 A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving an initial search query that includes (i) one or more query terms associated with an object, and (ii) one or more query terms that identify a context in which the object is to appear in images that are identified in response to the initial search query; determining a category associated with the object; generating a generalized search query that includes (i) one or more query terms associated with the category, and (ii) one or query terms that identify the context in which the object is to appear in images that are identified in response to the initial search query; obtaining (i) an initial set of one or more images that are identified as responsive to the initial search query, and (ii) a generalized set of one or more images that are identified as responsive to the generalized search query; and selecting one or more of the images of the initial set based at least on an indication of similarity between the selected one or more images and the one or more images of the generalized set of one or more images. ### Claim 9 (depends on 8) The system of claim 8, wherein the context comprises a viewpoint. ### Claim 10 (depends on 8) The system of claim 8, wherein determining a category associated with the object comprises identifying categories associated with the one or more query terms associated with the object in the initial search query. ### Claim 11 (depends on 8) The system of claim 8, the operations comprising generating the indication of similarity based on: determining a function based on the generalized set of one or more images and the first query; and determining to generate the indication of similarity based on evaluating the function using the selected one or more images. ### Claim 12 (depends on 8) The system of claim 8, the operations comprising generating the indication of similarity based on: training an image classifier using the one or more images of the generalized set of one or more images; and determining to generate the indication of similarity based on classifying the selected one or more images using the trained image classifier. ### Claim 13 (depends on 8) The system of claim 8, the operations comprising generating the indication of similarity based on: generating first representations of the selected one or more images; generating a second representation based on the generalized set of one or more images; identifying differences between the first representations and the second representation; and determining to generate the indication of similarity based on the identified differences. ### Claim 14 (depends on 8) The system of claim 8, the operations comprising: identifying, in the initial search query, the one or more query terms associated with the object; and identifying, in the initial search query, the one or more query terms that identify the context in which the object is to appear in images that are identified in response to the initial search query. ### Claim 15 A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising: receiving an initial search query that includes (i) one or more query terms associated with an object, and (ii) one or more query terms that identify a context in which the object is to appear in images that are identified in response to the initial search query; determining a category associated with the object; generating a generalized search query that includes (i) one or more query terms associated with the category, and (ii) one or query terms that identify the context in which the object is to appear in images that are identified in response to the initial search query; obtaining (i) an initial set of one or more images that are identified as responsive to the initial search query, and (ii) a generalized set of one or more images that are identified as responsive to the generalized search query; and selecting one or more of the images of the initial set based at least on an indication of similarity between the selected one or more images and the one or more images of the generalized set of one or more images. ### Claim 16 (depends on 15) The medium of claim 15, wherein the context comprises a viewpoint. ### Claim 17 (depends on 15) The medium of claim 15, wherein determining a category associated with the object comprises identifying categories associated with the one or more query terms associated with the object in the initial search query. ### Claim 18 (depends on 15) The medium of claim 15, the operations comprising generating the indication of similarity based on: determining a function based on the generalized set of one or more images and the first query; and determining to generate the indication of similarity based on evaluating the function using the selected one or more images. ### Claim 19 (depends on 15) The medium of claim 15, the operations comprising generating the indication of similarity based on: training an image classifier using the one or more images of the generalized set of one or more images; and determining to generate the indication of similarity based on classifying the selected one or more images using the trained image classifier. ### Claim 20 (depends on 15) The medium of claim 15, the operations comprising generating the indication of similarity based on: generating first representations of the selected one or more images; generating a second representation based on the generalized set of one or more images; identifying differences between the first representations and the second representation; and determining to generate the indication of similarity based on the identified differences.