# US 2018/0322654 A1 · Location determination using street view images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322654 A1 - Document ID: US20180322654A1 - Status: Published application - Kind: A1 - Application number: 15/585,721 - Filed: 2017-05-03 - Published: 2018-11-08 - Earliest claimed: 2017-05-03 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Xiao Long Chen; Na Lv; Zhe Yan; Yun Zhai; Zhuo Zhao - Web page: https://patentgrep.com/patent/US20180322654A1 ## CPC - G06T 7/74: {involving reference images or patches} - G06T 2207/20072: Graph-based image processing - G06T 2207/30244: Camera pose - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - H04M 1/724: User interfaces specially adapted for cordless or mobile telephones - H04M 1/72403: with means for local support of applications that increase the functionality - H04W 88/02: Terminal devices - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract A method, a computer program product, and a computer system for determining a location using street view images. A mobile device obtains an image captured by the mobile device, obtains a set of street view images, and obtains a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images. The first graph includes nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs includes nodes representing interest points in the set of the street view images. The mobile device determines a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs. ## Claims ### Claim 1 A method comprising: obtaining, by a mobile device, an image captured by the mobile device; obtaining, by the mobile device, a set of street view images; obtaining, by the mobile device, a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images, the first graph including nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs including nodes representing interest points in the set of the street view images; and determining, by the mobile device, a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining a location of the mobile device comprises: generating, by the mobile device, a sub-graph from one of the plurality of the second graphs; rotating, by the mobile device, the sub-graph by a plurality of angles to generate a plurality of sub-graphs; and determining, by the mobile device, relevance between the first graph and the plurality of the sub-graphs; and determining, by the mobile device, the location of the mobile device, based on the relevance between the first graph and the plurality of the sub-graphs. ### Claim 3 (depends on 1) The method of claim 1, wherein the obtaining the first graph of the image captured by the mobile device and the plurality of the second graphs of the set of the street view images comprises: identifying, by the mobile device, a plurality of interest points in the image captured by the mobile device and the set of the street view images; determining, by the mobile device, a plurality of edges between a plurality of nodes representing the plurality of the interest points; and generating, by the mobile device, the first graph and the plurality of the second graphs, based on the plurality of nodes and the plurality of edges. ### Claim 4 (depends on 3) The method of claim 3, wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of interest points comprises: determining, by the mobile device, a horizontal value, based on a horizontal position relationship between two nodes; determining, by the mobile device, a vertical value based on a vertical position relationship between the two nodes; and determining, by the mobile device, an edge value representing a position relationship between the two nodes, based on the horizontal value and the vertical value. ### Claim 5 (depends on 3) The method of claim 3, wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of the interest points comprises: determining, by the mobile device, a first set of edge values, each of which representing a position relationship between two of the plurality of the nodes in the first graph; and determining, by the mobile device, a second set of edge values, each of which representing a position relationship between two of the plurality of the nodes in one of the plurality of the second graphs. ### Claim 6 (depends on 5) The method of claim 5, wherein the determining the location of the mobile device comprises: determining, by the mobile device, a match degree between the first set of the edge values and the second set of the edge values; and in response to determining that the match degree exceeds a predefined threshold, obtaining, by the mobile device, a location of the one of the plurality of the second graphs, and determining, by the mobile device, the location of the one of the plurality of the second graphs as the location of the mobile device. ### Claim 7 (depends on 6) The method of claim 6, wherein the determining the match degree between the first set of the edge values and the second set of the edge values comprises: dividing, by the mobile device, nodes in the first graph and the plurality of second graphs into a plurality of groups according to an image feature of each node; and determining, by the mobile device, a match degree between a first edge value in the first set of the edge values and a second edge value in the second set of the edge values, first two nodes associated with the first edge value and second two nodes associated with the second edge value belonging to a same group of the plurality of groups. ### Claim 8 (depends on 1) The method of claim 1, further comprising: displaying, by the mobile device, the location of the mobile device on a map application; and providing, by the mobile device, a navigational recommendation to a destination on the map application. ### Claim 9 (depends on 1) The method of claim 1, the obtaining the set of the street view images comprises: determining, by the mobile device, geographic information of the mobile device; and obtaining, by the mobile device, the set of the street view images based on the geographic information. ### Claim 10 A computer system comprising: a processing unit; a memory coupled to the processing unit and storing instructions thereon, the instructions, when executed by the processing unit, performing acts including: obtaining, by a mobile device, an image captured by the mobile device; obtaining, by the mobile device, a set of street view images; obtaining, by the mobile device, a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images, the first graph including nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs including nodes representing interest points in the set of the street view images; and determining, by the mobile device, a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs. ### Claim 11 (depends on 10) The computer system of claim 10, wherein the determining the location of the mobile device comprises: generating, by the mobile device, a sub-graph from one of the plurality of the second graphs; rotating, by the mobile device, the sub-graph by a plurality of angles to generate a plurality of sub-graphs; and determining, by the mobile device, relevance between the first graph and the plurality of the sub-graphs; and determining, by the mobile device, the location of the mobile device, based on the relevance between the first graph and the plurality of the sub-graphs. ### Claim 12 (depends on 10) The computer system of claim 10, wherein the obtaining the first graph of the image captured by the mobile device and the plurality of the second graphs of the set of the street view images comprises: identifying, by the mobile device, a plurality of interest points in the image captured by the mobile device and the set of the street view images; determining, by the mobile device, a plurality of edges between a plurality of nodes representing the plurality of the interest points; and generating, by the mobile device, the first graph and the plurality of the second graphs, based on the plurality of nodes and the plurality of edges. ### Claim 13 (depends on 12) The computer system of claim 12, wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of interest points comprises: determining, by the mobile device, a horizontal value, based on a horizontal position relationship between two nodes; determining, by the mobile device, a vertical value based on a vertical position relationship between the two nodes; and determining, by the mobile device, an edge value representing a position relationship between the two nodes, based on the horizontal value and the vertical value. ### Claim 14 (depends on 12) The computer system of claim 12, wherein the determining the plurality of the edges between the plurality of the nodes representing the plurality of the interest points comprises: determining, by the mobile device, a first set of edge values, each of which representing a position relationship between two of the plurality of the nodes in the first graph; and determining, by the mobile device, a second set of edge values, each of which representing a position relationship between two of the plurality of the nodes in one of the plurality of the second graphs. ### Claim 15 (depends on 14) The computer system of claim 14, wherein the determining the location of the mobile device comprises: determining, by the mobile device, a match degree between the first set of the edge values and the second set of the edge values; and in response to determining that the match degree exceeds a predefined threshold, obtaining, by the mobile device, a location of the one of the plurality of the second graphs, and determining, by the mobile device, the location of the one of the plurality of the second graphs as the location of the mobile device. ### Claim 16 (depends on 15) The computer system of claim 15, wherein the determining the match degree between the first set of the edge values and the second set of the edge values comprises: dividing, by the mobile device, nodes in the first graph and the plurality of second graphs into a plurality of groups according to an image feature of each node; and determining, by the mobile device, a match degree between a first edge value in the first set of the edge values and a second edge value in the second set of the edge values, first two nodes associated with the first edge value and second two nodes associated with the second edge value belonging to a same group of the plurality of groups. ### Claim 17 (depends on 10) The computer system of claim 10, the instructions, when executed by the processing unit, further performing acts including: displaying, by the mobile device, the location of the mobile device on a map application; and providing, by the mobile device, a navigational recommendation to a destination on the map application. ### Claim 18 (depends on 10) The computer system of claim 10, the instructions, when executed by the processing unit, further performing acts including: determining, by the mobile device, geographic information of the mobile device; and obtaining, by the mobile device, the set of the street view images based on the geographic information. ### Claim 19 A computer program product being tangibly stored on a non-transient machine-readable medium and comprising machine-executable instructions, the instructions, when executed on a mobile device, causing the mobile device to perform: obtaining, by the mobile device, an image captured by the mobile device; obtaining, by the mobile device, a set of street view images; obtaining, by the mobile device, a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images, the first graph including nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs including nodes representing interest points in the set of the street view images; and determining, by the mobile device, a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs. ### Claim 20 (depends on 19) The computer program product of claim 19, wherein the determining the location of the mobile device comprises: generating, by the mobile device, a sub-graph from one of the plurality of the second graphs; rotating, by the mobile device, the sub-graph by a plurality of angles to generate a plurality of sub-graphs; and determining, by the mobile device, relevance between the first graph and the plurality of the sub-graphs; and determining, by the mobile device, the location of the mobile device, based on the relevance between the first graph and the plurality of the sub-graphs.