# US 2017/0286493 A1 · Hybrid Use of Location Sensor Data And Visual Query to Return Local Listings For Visual Query

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286493 A1
- Document ID: US20170286493A1
- Status: Published application
- Kind: A1
- Application number: 15/624,056
- Filed: 2017-06-15
- Published: 2017-10-05
- Earliest claimed: 2009-12-03
- Assignees: Google Inc.
- Applicants: Google Inc.
- Inventors: David Petrou; John Flynn; Hartwig Adam; Hartmut Neven
- Web page: https://patentgrep.com/patent/US20170286493A1

## CPC

- G06F 16/583: using metadata automatically derived from the content
- G06F 16/242: Query formulation
- G06F 16/24575: {using context}
- G06F 16/29: Geographical information databases
- H04W 4/02: Services making use of location information
- H04W 4/029: Location-based management or tracking services
- H04W 64/00: Locating users or terminals {or network equipment} for network management purposes, e.g. mobility management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A visual query is received from a client system, along with location information for the client system, and processed by a server system. The server system sends the visual query and the location information to a visual query search system, and receives from the visual query search system enhanced location information based on the visual query and the location information. The server system then sends a search query, including the enhanced location information, to a location-based search system. The search system receives and provides to the client one or more search results to the client system.

## Claims

### Claim 1

(canceled)

### Claim 2

A computer-implemented method for identifying similar documents, comprising:
receiving an image query from a client system, the image query including an image;
identifying a location of the client system;
determining an accuracy value that specifies a level of accuracy for the identified location;
identifying a plurality of search results based on the received image query and the identified location, each search result having an associated prominence value that specifies a relative importance of the search result and a closeness metric that specifies how close a location of an entity referenced by the search result is to the location of the client system;
determining whether the accuracy value satisfies a threshold;
selecting one or more of the plurality of search results based on the prominence value and the closeness metric for each search result of the plurality of search results, wherein the closeness metric for each search result is given more weight than the prominence value for each search result in selecting the one or more search results in response to determining that the accuracy value satisfies the threshold; and
providing the one or more selected search results to the client system.

### Claim 3 (depends on 2)

The method of claim 2, wherein the closeness metric for each search result is given less weight than the prominence value for each search result in selecting the one or more search results in response to determining that the accuracy value does not satisfy the threshold.

### Claim 4 (depends on 2)

The method of claim 2, wherein selecting the one or more search results comprises:
selecting a first weighting factor for each prominence value based on the accuracy value;
selecting a second weighting factor for each closeness metric based on the accuracy value;
weighting, using the first weighting factor, the prominence value for each search result of the plurality of search results;
weighting, using the second weighting factor, the closeness metric for each search result of the plurality of search results; and
selecting the one or more search results based on the weighted prominence values and the weighted closeness metrics.

### Claim 5 (depends on 2)

The method of claim 2, wherein identifying the plurality of search results comprises:
for each particular search result of a set of search results:
identifying a location of a particular entity referenced by the particular search result; and
identifying a positional accuracy value that specifies a level of accuracy for the identified location of the particular entity; and
selecting the plurality of search results from the set of search results based on the positional accuracy value for each particular search result.

### Claim 6 (depends on 5)

The method of claim 5, wherein selecting the plurality of search results from the set of search results comprises:
identifying a closeness metric for each particular search result, the closeness metric for a particular search result specifying how close the entity referenced by the particular search result is to the location of the client system; and
selecting, for inclusion in the plurality of search results, each particular search result that has a closeness metric that satisfies a closeness threshold and a positional accuracy that satisfies an accuracy threshold.

### Claim 7 (depends on 6)

The method of claim 6, wherein selecting the plurality of search results from the set of search results comprises:
identifying, for inclusion in a proper subset of search results, each search result of the set of search results that has a positional accuracy that does not satisfy the accuracy threshold; and
selecting, for inclusion in the plurality of search results, each search result of the proper subset that references an entity that is located within a threshold distance of an entity referenced by a search result that has a closeness metric that satisfies the closeness threshold and a positional accuracy that satisfies the accuracy threshold.

### Claim 8 (depends on 2)

The method of claim 2, wherein identifying the location of the client system comprises:
obtaining location information from the client system, the location information specifying a first location of the client system;
comparing the image of the image query with a plurality of images of entities located within a predefined range of the first location of the client;
determining an enhanced location of the client system using image location information specifying a location of an entity depicted in at least one of the plurality of image; and
designating the enhanced location of the client system as the location of the client system.

### Claim 9

A system, comprising:
a data processing apparatus; and
a memory storage apparatus in data communication with the data processing apparatus, the memory storage apparatus storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising:
receiving an image query from a client system, the image query including an image;
identifying a location of the client system;
determining an accuracy value that specifies a level of accuracy for the identified location;
identifying a plurality of search results based on the received image query and the identified location, each search result having an associated prominence value that specifies a relative importance of the search result and a closeness metric that specifies how close a location of an entity referenced by the search result is to the location of the client system;
determining whether the accuracy value satisfies a threshold;
selecting one or more of the plurality of search results based on the prominence value and the closeness metric for each search result of the plurality of search results, wherein the closeness metric for each search result is given more weight than the prominence value for each search result in selecting the one or more search results in response to determining that the accuracy value satisfies the threshold; and
providing the one or more selected search results to the client system.

### Claim 10 (depends on 9)

The system of claim 9, wherein the closeness metric for each search result is given less weight than the prominence value for each search result in selecting the one or more search results in response to determining that the accuracy value does not satisfy the threshold.

### Claim 11 (depends on 9)

The system of claim 9, wherein selecting the one or more search results comprises:
selecting a first weighting factor for each prominence value based on the accuracy value;
selecting a second weighting factor for each closeness metric based on the accuracy value;
weighting, using the first weighting factor, the prominence value for each search result of the plurality of search results;
weighting, using the second weighting factor, the closeness metric for each search result of the plurality of search results; and
selecting the one or more search results based on the weighted prominence values and the weighted closeness metrics.

### Claim 12 (depends on 9)

The system of claim 9, wherein identifying the plurality of search results comprises:
for each particular search result of a set of search results:
identifying a location of a particular entity referenced by the particular search result; and
identifying a positional accuracy value that specifies a level of accuracy for the identified location of the particular entity; and
selecting the plurality of search results from the set of search results based on the positional accuracy value for each particular search result.

### Claim 13 (depends on 9)

The system of claim 9, wherein selecting the plurality of search results from the set of search results comprises:
identifying a closeness metric for each particular search result, the closeness metric for a particular search result specifying how close the entity referenced by the particular search result is to the location of the client system; and
selecting, for inclusion in the plurality of search results, each particular search result that has a closeness metric that satisfies a closeness threshold and a positional accuracy that satisfies an accuracy threshold.

### Claim 14 (depends on 13)

The system of claim 13, wherein selecting the plurality of search results from the set of search results comprises:
identifying, for inclusion in a proper subset of search results, each search result of the set of search results that has a positional accuracy that does not satisfy the accuracy threshold; and
selecting, for inclusion in the plurality of search results, each search result of the proper subset that references an entity that is located within a threshold distance of an entity referenced by a search result that has a closeness metric that satisfies the closeness threshold and a positional accuracy that satisfies the accuracy threshold.

### Claim 15 (depends on 9)

The system of claim 9, wherein identifying the location of the client system comprises:
obtaining location information from the client system, the location information specifying a first location of the client system;
comparing the image of the image query with a plurality of images of entities located within a predefined range of the first location of the client;
determining an enhanced location of the client system using image location information specifying a location of an entity depicted in at least one of the plurality of image; and
designating the enhanced location of the client system as the location of the client system.

### Claim 16

A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations comprising:
receiving an image query from a client system, the image query including an image;
identifying a location of the client system;
determining an accuracy value that specifies a level of accuracy for the identified location;
identifying a plurality of search results based on the received image query and the identified location, each search result having an associated prominence value that specifies a relative importance of the search result and a closeness metric that specifies how close a location of an entity referenced by the search result is to the location of the client system;
determining whether the accuracy value satisfies a threshold;
selecting one or more of the plurality of search results based on the prominence value and the closeness metric for each search result of the plurality of search results, wherein the closeness metric for each search result is given more weight than the prominence value for each search result in selecting the one or more search results in response to determining that the accuracy value satisfies the threshold; and
providing the one or more selected search results to the client system.

### Claim 17 (depends on 16)

The computer storage medium of claim 16, wherein the closeness metric for each search result is given less weight than the prominence value for each search result in selecting the one or more search results in response to determining that the accuracy value does not satisfy the threshold.

### Claim 18 (depends on 16)

The computer storage medium of claim 16, wherein selecting the one or more search results comprises:
selecting a first weighting factor for each prominence value based on the accuracy value;
selecting a second weighting factor for each closeness metric based on the accuracy value;
weighting, using the first weighting factor, the prominence value for each search result of the plurality of search results;
weighting, using the second weighting factor, the closeness metric for each search result of the plurality of search results; and
selecting the one or more search results based on the weighted prominence values and the weighted closeness metrics.

### Claim 19 (depends on 16)

The computer storage medium of claim 16, wherein identifying the plurality of search results comprises:
for each particular search result of a set of search results:
identifying a location of a particular entity referenced by the particular search result; and
identifying a positional accuracy value that specifies a level of accuracy for the identified location of the particular entity; and
selecting the plurality of search results from the set of search results based on the positional accuracy value for each particular search result.

### Claim 20 (depends on 19)

The computer storage medium of claim 19, wherein selecting the plurality of search results from the set of search results comprises:
identifying a closeness metric for each particular search result, the closeness metric for a particular search result specifying how close the entity referenced by the particular search result is to the location of the client system; and
selecting, for inclusion in the plurality of search results, each particular search result that has a closeness metric that satisfies a closeness threshold and a positional accuracy that satisfies an accuracy threshold.

### Claim 21 (depends on 20)

The computer storage medium of claim 20, wherein selecting the plurality of search results from the set of search results comprises:
identifying, for inclusion in a proper subset of search results, each search result of the set of search results that has a positional accuracy that does not satisfy the accuracy threshold; and
selecting, for inclusion in the plurality of search results, each search result of the proper subset that references an entity that is located within a threshold distance of an entity referenced by a search result that has a closeness metric that satisfies the closeness threshold and a positional accuracy that satisfies the accuracy threshold.
