# US 2015/0178321 A1 · Image-based 3d model search and retrieval > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2015/0178321 A1 - Document ID: US20150178321A1 - Status: Published application - Kind: A1 - Application number: 13/796,936 - Filed: 2013-03-12 - Published: 2015-06-25 - Earliest claimed: 2012-04-10 - Assignees: Google Inc. - Inventors: Ehud Rivlin; Maks Ovsjanikov - Web page: https://patentgrep.com/patent/US20150178321A1 ## CPC - G06F 16/532: Query formulation, e.g. graphical querying - G06F 16/56: having vectorial format - G06F 16/583: using metadata automatically derived from the content - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A query may be associated with 3D models based on the relevancy of each 3D model to the query. For a given query, a set of views are selected from a 3D model. Relevance scores are assigned to the views based on image features that are shared with popular on-line images that are accessed based on the query. The relevance scores of the views are aggregated to provide the 3D model with a relevance score. In response to a user search for a 3D model, the model having the highest relevance score for a query that matches the search is selected and returned to the user. The 3D model may be displayed with the 2D view that has the highest relevance score. In some cases, a set of 3D models having the highest relevance scores is returned to the user such that the user may select an appropriate 3D model. ## Claims ### Claim at position 1 1-22. (canceled) ### Claim 23 A computer-implemented method comprising: receiving a search query; selecting multiple, different two-dimensional views of a three-dimensional model that is identified as responsive to the search query; generating an image query that includes a query term that matches a query term of the search query; obtaining images that are identified as response to the image query that includes the query term that matches the query term of the search query; generating a respective score for each one of the multiple, different two-dimensional views based on one or more similarities between the views and the images that are identified as response to the image query that includes the query term that matches the query term of the search query; selecting a particular two-dimensional view of the three-dimensional model from among the multiple, different two-dimensional views as a representative view of the three-dimensional model based on the scores of the multiple, different two-dimensional views; and providing the representative view of the three-dimensional model in response to the search query. ### Claim 24 (depends on 23) The method of claim 23, comprising: identifying the three-dimensional model that is responsive to the search query based on a keyword match between a query term of the search query and a tag of the model. ### Claim 25 (depends on 23) The method of claim 23, comprising: generating the multiple, different two-dimensional views by sampling the three-dimensional model at different view points. ### Claim 26 (canceled) ### Claim 27 (depends on 23) The method of claim 23, wherein generating a respective score for each one of the multiple, different two-dimensional views based on one or more similarities between the views and the images that are identified as response to the image query that includes the query term that matches the query term of the search query comprises: extracting view features from the multiple, different two-dimensional views; extracting image features from the images that are identified as responsive; and for each of the multiple different two-dimensional views, generating the respective score based on comparing the extracted view features of the view with the extracted image features. ### Claim 28 (depends on 23) The method of claim 23, wherein selecting the particular two-dimensional view of the three-dimensional model from among the multiple, different two-dimensional views as a representative view of the three-dimensional model based on the scoring of the multiple, different two-dimensional views comprises: determining the particular two-dimensional view of the three-dimensional model has the highest score, indicating that the particular two-dimensional view is most similar to the images that are identified as responsive to the image query; and in response to the determination, selecting the particular two-dimensional view of the three-dimensional model. ### Claim 29 (depends on 23) The method of claim 23, wherein providing the representative view of the three-dimensional model in response to the search query comprises: identifying the three-dimensional model is response to the search query based on the scoring of the multiple, different two-dimensional views of the three-dimensional model; and in response to identifying the three-dimensional model is response to the search query, providing the representative view of the three-dimensional model in response to the search query. ### Claim 30 A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving a search query; selecting multiple, different two-dimensional views of a three-dimensional model that is identified as responsive to the search query; generating an image query that includes a query term that matches a query term of the search query; obtaining images that are identified as response to the image query that includes the query term that matches the query term of the search query; generating a respective score for each one of the multiple, different two-dimensional views based on one or more similarities between the views and the images that are identified as response to the image query that includes the query term that matches the query term of the search query; selecting a particular two-dimensional view of the three-dimensional model from among the multiple, different two-dimensional views as a representative view of the three-dimensional model based on the scores of the multiple, different two-dimensional views; and providing the representative view of the three-dimensional model in response to the search query. ### Claim 31 (depends on 30) The system of claim 30, wherein the operations comprise: identifying the three-dimensional model that is responsive to the search query based on a keyword match between a query term of the search query and a tag of the model. ### Claim 32 (depends on 30) The system of claim 30, wherein the operations comprise: generating the multiple, different two-dimensional views by sampling the three-dimensional model at different view points. ### Claim 33 (canceled) ### Claim 34 (depends on 30) The system of claim 30, wherein generating a respective score for each one of the multiple, different two-dimensional views based on one or more similarities between the views and the images that are identified as response to the image query that includes the query term that matches the query term of the search query comprises: extracting view features from the multiple, different two-dimensional views; extracting image features from the images that are identified as responsive; and for each of the multiple different two-dimensional views, generating the respective score based on comparing the extracted view features of the view with the extracted image features. ### Claim 35 (depends on 30) The system of claim 30, wherein selecting the particular two-dimensional view of the three-dimensional model as a representative view of the three-dimensional model based on the scoring of the multiple, different two-dimensional views comprises: determining the particular two-dimensional view of the three-dimensional model has the highest score, indicating that the particular two-dimensional view is most similar to the images that are identified as responsive to the image query; and in response to the determination, selecting the particular two-dimensional view of the three-dimensional model. ### Claim 36 (depends on 30) The system of claim 30, wherein providing the representative view of the three-dimensional model in response to the search query comprises: identifying the three-dimensional model is response to the search query based on the scoring of the multiple, different two-dimensional views of the three-dimensional model; and in response to identifying the three-dimensional model is response to the search query, providing the representative view of the three-dimensional model in response to the search query. ### Claim 37 A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising: receiving a search query; selecting multiple, different two-dimensional views of a three-dimensional model that is identified as responsive to the search query; generating an image query that includes a query term that matches a query term of the search query; obtaining images that are identified as response to the image query that includes the query term that matches the query term of the search query; generating a respective score for each one of the multiple, different two-dimensional views based on one or more similarities between the views and the images that are identified as response to the image query that includes the query term that matches the query term of the search query; selecting a particular two-dimensional view of the three-dimensional model from among the multiple, different two-dimensional views as a representative view of the three-dimensional model based on the scores of the multiple, different two-dimensional views; and providing the representative view of the three-dimensional model in response to the search query. ### Claim 38 (depends on 37) The medium of claim 37, wherein the operations comprise: identifying the three-dimensional model that is responsive to the search query based on a keyword match between a query term of the search query and a tag of the model. ### Claim 39 (depends on 37) The medium of claim 37, wherein the operations comprise: generating the multiple, different two-dimensional views by sampling the three-dimensional model at different view points. ### Claim 40 (canceled) ### Claim 41 (depends on 37) The medium of claim 37, wherein generating a respective score for each one of the multiple, different two-dimensional views based on one or more similarities between the views and the images that are identified as response to the image query that includes the query term that matches the query term of the search query comprises: extracting view features from the multiple, different two-dimensional views; extracting image features from the images that are identified as responsive; and for each of the multiple different two-dimensional views, generating the respective score based on comparing the extracted view features of the view with the extracted image features. ### Claim 42 (depends on 37) The medium of claim 37, wherein selecting the particular two-dimensional view of the three-dimensional model as a representative view of the three-dimensional model based on the scoring of the multiple, different two-dimensional views comprises: determining the particular two-dimensional view of the three-dimensional model has the highest score, indicating that the particular two-dimensional view is most similar to the images that are identified as responsive to the image query; and in response to the determination, selecting the particular two-dimensional view of the three-dimensional model. ### Claim 43 (depends on 37) The medium of claim 37, wherein providing the representative view of the three-dimensional model in response to the search query comprises: identifying the three-dimensional model is response to the search query based on the scoring of the multiple, different two-dimensional views of the three-dimensional model; and in response to identifying the three-dimensional model is response to the search query, providing the representative view of the three-dimensional model in response to the search query.