# US 2012/0254144 A1 · Using network traffic logs for search engine index updates

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0254144 A1
- Document ID: US20120254144A1
- Status: Published application
- Kind: A1
- Application number: 13/492,508
- Filed: 2012-06-08
- Published: 2012-10-04
- Earliest claimed: 2001-04-16
- Assignees: Yahoo! Inc.
- Inventors: Arkady Borkovsky; Douglas M. Cook; Jean-Marc Langlois; Tomi Poutanen; Hongyuan Zha
- Web page: https://patentgrep.com/patent/US20120254144A1

## CPC

- G06F 16/951: Indexing; Web crawling techniques
- H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)}
- H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching
- Y10S 707/99933: Query processing, i.e. searching
- Y10S 707/99937: Sorting
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

A method and apparatus for using network traffic logs for search enhancement is disclosed. According to one embodiment, network usage is tracked by generating log files. These log files among other things indicate the frequency web pages are referenced and modified. These log files or information from these log files can then be used to improve document ranking, improve web crawling, determine tiers in a multi-tiered index, determine where to insert a document in a multi-tiered index, determine link weights, and update a search engine index.

## Claims

### Claim 1

A method comprising:
receiving network usage information from a traffic server, wherein the network usage information identifies one or more information sources that have been modified;
in response to receiving the network usage information, retrieving information from at least a particular information source of the one or more information sources and updating a search engine index based on the information;
wherein the method is performed by one or more computing devices.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more information sources are web pages.

### Claim 3 (depends on 2)

The method of claim 2, wherein the traffic server compiles the network usage information based at least in part on a Last-Modified field of one or more Hypertext Transfer Protocol (HTTP) headers that each identifies at least one information source of the one or more information sources.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
in response to receiving the network usage information, updating a modification frequency log indicating at least how frequently each of the one or more information sources have been modified.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
selecting the particular information source from a plurality of information sources represented in the search engine index based at least in part on the modification frequency log.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
determining how frequently to access the particular information source to update the search engine index based at least in part on the frequency log.

### Claim 7 (depends on 1)

The method of claim 1, wherein the network usage information indicates when each of the one or more information sources was last modified and the method further comprising:
selecting the particular information source from a plurality of information sources represented in the search engine index in response to a determination that the particular information source was last accessed to update the index prior to when the particular information source was last modified.

### Claim 8 (depends on 1)

The method of claim 1, wherein the traffic server, in response to receiving a request for an information source, determines whether the information source has been modified within a specified length of time.

### Claim 9

A non-transitory computer-readable storage medium storing one or more sequences of instructions that when executed by one or more processors causes the one or more processors to perform the steps of:
receiving network usage information from a traffic server, wherein the network usage information identifies one or more information sources that have been modified;
in response to receiving the network usage information, retrieving information from at least a particular information source of the one or more information sources and updating a search engine index based on the information.

### Claim 10 (depends on 9)

The non-transitory computer-readable storage medium of claim 9, wherein the one or more information sources are web pages.

### Claim 11 (depends on 10)

The non-transitory computer-readable storage medium of claim claim 10, wherein the traffic server compiles the network usage information based at least in part on a Last-Modified field of one or more Hypertext Transfer Protocol (HTTP) headers that each identifies at least one information source of the one or more information sources.

### Claim 12 (depends on 9)

The non-transitory computer-readable storage medium of claim 9, further comprising instructions for:
in response to receiving the network usage information, updating a modification frequency log indicating at least how frequently each of the one or more information sources have been modified.

### Claim 13 (depends on 12)

The non-transitory computer-readable storage medium of claim 12, further comprising instructions for:
selecting the particular information source from a plurality of information sources represented in the search engine index based at least in part on the modification frequency log.

### Claim 14 (depends on 12)

The non-transitory computer-readable storage medium of claim 12, further comprising instructions for:
determining how frequently to access the particular information source to update the search engine index based at least in part on the frequency log.

### Claim 15 (depends on 9)

The non-transitory computer-readable storage medium of claim 9, wherein the network usage information indicates when each of the one or more information sources was last modified and the computer-readable medium further comprising instructions for:
selecting the particular information source from a plurality of information sources represented in the search engine index in response to a determination that the particular information source was last accessed to update the index prior to when the particular information source was last modified.

### Claim 16 (depends on 9)

The non-transitory computer-readable storage medium of claim 9, wherein the traffic server, in response to receiving a request for an information source, determines whether the information source has been modified within a specified length of time.
