# US 8,788,504 B1 · Web mining to build a landmark database and applications thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,788,504 B1
- Document ID: US8788504B1
- Status: Granted
- Kind: B1
- Application number: 12/269,445
- Filed: 2008-11-12
- Granted: 2014-07-22
- Earliest claimed: 2008-11-12
- Assignees: Google Inc.
- Applicants: Ming Zhao; Yantao Zheng; Yang Song; Hartwig Adams
- Inventors: Ming Zhao; Yantao Zheng; Yang Song; Hartwig Adams
- Web page: https://patentgrep.com/patent/US8788504B1

## CPC

- G06F 16/21: Design, administration or maintenance of databases
- G06F 16/22: Indexing; Data structures therefor; Storage structures
- G06F 16/58: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/583: using metadata automatically derived from the content
- G06F 16/95: Retrieval from the web
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

This invention relates to building a landmark database from web data. In one embodiment, a computer-implemented method builds a landmark database. Web data including a web page is received from one or more websites via one or more networks. The web data is interpreted using at least one processor to determine landmark data describing a landmark. At least a portion of the landmark data identifies a landmark. Finally, a visual model is generated using the landmark data. A computing device is able to recognize the landmark in an image based on the visual model.

## Claims

### Claim 1

A computer-implemented method for building a landmark database, comprising:
(a) receiving web data including a web page from one or more websites via one or more networks;
(b) interpreting the web data using at least one processor to extract landmark data describing a landmark from the web data, the landmark data including a name that identifies the landmark and a portion of a street address where the landmark is located;
(c) applying one or more error checking rules to remove inaccurate landmark data from the landmark data;
(d) generating a visual model using images determined based on the landmark data including the name;
(e) sending, to a geocoding service, a request with the name of the landmark and the portion of the street address where the landmark is located; and
(f) in response to the request, a location of the landmark from the geocoding service, the location determined based on the name of the landmark and the portion of the street address where the landmark is located included in the request,
whereby a computing device is able to, using the visual model, recognize the landmark in an image, different from the images determined based on the landmark data, and correlate the image with the name.

### Claim 2 (depends on 1)

The method of claim 1, wherein the interpreting the web data of step (b) comprises parsing the web data to determine the landmark data according to one or more extraction rules.

### Claim 3 (depends on 1)

The method of claim 1, wherein the generating the visual model of step (d) comprises:
(i) retrieving a set of results from a search engine, wherein at least one result in the set of results includes an image of the landmark; and
(ii) constructing a visual model based on images of the landmark in the set of results, wherein a computing device is able to recognize the landmark in an image using the visual model.

### Claim 4 (depends on 3)

The method of claim 3, wherein the generating the visual model step (d) further comprises:
(iii) applying a detector to determine that the result in the set of results does not include an image of the landmark.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
(g) storing the location, the visual model, and a portion of the landmark data identifying the landmark in a database of landmarks.

### Claim 6 (depends on 1)

The method of claim 1, wherein a computing device is able to recognize the landmark in an image from a frame of a video based on the visual model.

### Claim 7

A system for building a landmark database, comprising:
a web data retriever module configured to receive web data including a web page from one or more websites via one or more networks;
a web data interpreter module configured to interpret the web data to extract landmark data describing a landmark, wherein the landmark data comprises a name that identifies the landmark and a portion of a street address where the landmark is located;
an error checker module configured to apply one or more error checking rules to remove inaccurate landmark data from the landmark data;
a visual model generator configured to generate a visual model using images determined based on the landmark data including the name; and
a geocoding module configured to send a request with the name of the landmark and the portion of a street address where the landmark is located to a geocoding service and to receive a location of the landmark from the geocoding service in response to the request, the location determined based on the name of the landmark and the portion of the street address included in the request,
wherein a computing device is able to, using the visual model, recognize the landmark in an image, different from the images determined based on the landmark data, and correlate the image with the name wherein the web data retriever module, the web data interpreter module, and the visual model generator are implemented on at least one computing device.

### Claim 8 (depends on 7)

The system of claim 7, wherein the web data interpreter module comprises a web data parser module configured to parse the web data to determine the landmark according to one or more extraction rules.

### Claim 9 (depends on 7)

The system of claim 7, wherein the visual model generator module comprises:
an image retriever module configured to retrieve a set of results from a search engine, wherein at least one result includes an image of the landmark; and
a visual clustering module configured to construct a visual model based on images of the landmark in the set of results, wherein a computing device is able to recognize the landmark using the visual model.

### Claim 10 (depends on 9)

The system of claim 9, wherein the visual model generator module further comprises:
an image checker module configured to apply a detector to determine that a result in the set of results does not include an image of the landmark.

### Claim 11 (depends on 7)

The system of claim 7, further comprising:
a landmark database that stores the location, the visual model, and a portion of the landmark data identifying the landmark in a database of landmarks.

### Claim 12 (depends on 7)

The system of claim 7, wherein a computing device is able to recognize the landmark in an image from a frame of a video based on the visual model.

### Claim 13

An apparatus comprising at least one non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by a computing device, cause the computer device to perform operations comprising:
receiving web data including a web page from one or more websites via one or more networks;
interpreting the web data using at least one processor to extract landmark data describing a landmark from the web data, the landmark data including a name that identifies the landmark and a portion of a street address where the landmark is located;
applying one or more error checking rules to remove inaccurate landmark data from the landmark data;
generating a visual model using images determined based on the landmark data including the name;
sending, to a geocoding service, a request with the name of the landmark and the portion of the street address where the landmark is located;
receiving a response to the request, the response including a location of the landmark from the geocoding service, the location determined based on the name of the landmark and the portion of the street address where the landmark is located included in the request;
using the visual model, recognizing the landmark in an image, different from the images determined based on the landmark data; and
correlating the image with the name.