Partitioning a search space for distributed crawling
AssigneeInternational Business Machines Corporation
InventorsGuy-Vincent Jourdan, losif Viorel Onut, Seyed M. Taheri, Gregor von Bochmann
Life of this patent
Estimating the term…
Loading patent…
AssigneeInternational Business Machines Corporation
InventorsGuy-Vincent Jourdan, losif Viorel Onut, Seyed M. Taheri, Gregor von Bochmann
Estimating the term…
An illustrative embodiment of a computer-implemented process for partitioning a crawling space computes an event identifier for each event in the set of events to form an identified set of events, segments the identified set of events into a number of partitions, assigns a partition to each node in a set of nodes and executes each event in each assigned partition by a respective node. In response to a determination that a new state is discovered, other nodes are notified of the new state, in which information associated with the new state is added to a respective assigned set of event IDs at each node. In response to a determination that no more notifications exist, the computer-implemented process determines whether more events to process exist and terminates in response to a determination that no more events to process exist.
A computer-implemented process for partitioning a crawling space, comprising:
A computer program product for partitioning a crawling space, comprising a storage medium embodied with machine-readable program instructions, which when executed by a computer causes the computer to implement a process, the process comprising:
An apparatus for partitioning a crawling space, the apparatus comprising:
Dependencies are inferred from the claim text.