Using network traffic logs for search engine index updates
AssigneeYahoo! Inc.
InventorsArkady Borkovsky, Douglas M. Cook, Jean-Marc Langlois, Tomi Poutanen, Hongyuan Zha
Life of this patent
Estimating the term…
Abstract
A method and apparatus for using network traffic logs for search enhancement is disclosed. According to one embodiment, network usage is tracked by generating log files. These log files among other things indicate the frequency web pages are referenced and modified. These log files or information from these log files can then be used to improve document ranking, improve web crawling, determine tiers in a multi-tiered index, determine where to insert a document in a multi-tiered index, determine link weights, and update a search engine index.
Claims14
A method comprising:
- determining whether an information source requested by a user has been modified within a specified length of time based on a Last-Modified field of a Hypertext Transfer Protocol (HTTP) header of a message associated with the information source;
- in response to determining that the information source has been modified within the specified length of time, creating an entry in a modified information source log that identifies the information source;
- retrieving information from the information source based on the modified information source log;
- and
- updating a search engine index based on the information;
- wherein the method is performed by one or more computing devices.
A non-transitory computer-readable storage medium storing one or more sequences of instructions that when executed by one or more processors causes the one or more processors to perform the steps of:
- determining whether an information source requested by a user has been modified within a specified length of time based on a Last-Modified field of a Hypertext Transfer Protocol (HTTP) header of a message associated with the information source;
- in response to determining that the information source has been modified within the specified length of time, creating an entry in a modified information source log that identifies the information source;
- retrieving information from the information source based on the modified information source log;
- and
- updating a search engine index based on the information.
Dependencies are inferred from the claim text.