US 6,701,317 B1 · Web page connectivity server construction

Abstract

A process for constructing a server for collecting, arranging and storing data that defines the connectivity of pages on the World Wide Web (Web). The process input is a set of compressed ASCII links files, wherein each links file is a series of source URLs and corresponding destination URLs. A temporary URLs_info Table is created and initialized. The links files and URLs metadata are read. Buffers of unique URLs are sorted and written from the links files into URL runs. An ID Index is created from the URL_info table. CS_ids are assigned to URLs and written to the ID Index. Both a compressed URL data structure and a URL Index are created. A Host Table is created. URL fingerprints are converted to CS_ids, and preliminary outstarts to CS_ids and preliminary outstarts and outlinks tables are created. Compressed outstarts and outlinks tables are created from the preliminary tables. Subsequently, compressed instarts and inlinks tables are created based on the outstarts and outlinks tables.

Claims33

Dependent claims

A process for constructing a Host Database comprising:

creating a preliminary Host Table in a connectivity server that collects, arranges and stores data to define the connectivity of pages on the Web,
sorting URLs in a URL database concurrently with the creating of the preliminary Host Table occurring;
and
creating a Host Table and a Host Index, the Host Table containing information about the URLs from the preliminary Host Table.

1 more element

An information storage device for storing, arranging and presenting data defining the connectivity of pages on the Web, the device comprising:

a URL Database that stores URLs and that associates a fingerprint and a CS_id with, each stored URL, the URL Database comprising:
a URL Database API,
a URL Index Array, and
a plurality of partitions that store URLs according to a predetermined criteria;
a Host Database that associates a Host_id with each distinct hostname in the URL Database, the host comprising:
a Host Database API that operates to accept a CS_id and return a corresponding Host_id and to accept a Host_id and return the CS_ids on the corresponding host;
and
a Link Database that stores links between source URLs and destination URLs, the link Database comprising:
a Link Database API that operates to retrieve, for a given CS_id, the number of outlinks from the URL corresponding to the CS_id and the number of inlinks to that URL, wherein the device is constructed in accordance with a method comprising:
(a) reading a set of links files;
(b) creating a temporary URLs_info Table;
(c) creating an B) Index from the URLs_info Table;
(d) assigning CS_ids to URLs;
(e) writing the CS_ids to the ID Index;
(f) compressing URLs;
(g) creating a URL Index;
(h) creating a Host Table;
(i) converting URL fingerprints to CS_ids;
(j) creating outstarts and outlinks tables;
and
(k) writing instarts and inlink tables to a partitioned URL Database.

18 more elements

Dependencies are inferred from the claim text.

Web page connectivity server construction

Life of this patent

Web page connectivity server construction

Life of this patent

Abstract

Claims33