Next generation near real-time indexing
AssigneeeBay Inc.
InventorsSwati Agarwal, Amit Basu, Thomas Pan, Pradeep Sankaranthi, Nicholas Whyte
Life of this patent
Estimating the term…
Abstract
Methods and systems to build and utilize a search infrastructure are described. The system generates index information, including document updates and indexes. The system receives event notifications as the document updates are received and accumulates the index information until published. A query engine receives a search query from a client machine and identifies search results based on the query and the index information. The system communicates the search results, over the network, to the client machine.
Claims16
A computer system to rebuild indexing information in near real time, the computer system comprising:
- processors;
- a plurality of query node servers;
- a non-transitory first indexing subsystem when executing on at least one processor among the processors cause the first indexing subsystem to generate full indexes to publish document updates by the plurality of query node servers via a distribution system, the first indexing system representing a map-reduced indexing system;
- a non-transitory second indexing subsystem when executing on at least one processor among the processors cause the second indexing subsystem to generate mini-indexes associated with the full indexes to publish document updates by the plurality of query node servers indirectly via the distribution system or directly to the plurality of query node servers, the second indexing subsystem representing a daemon-based indexing system that monitors for and receives in near real-time event notifications and generates the mini-indexes in near real-time based on receiving the event notifications, the event notifications based on a priority ordering of events by application servers;
- a non-transitory query engine when executing on at least one processor among the processors cause the computer system to:
- update the indexing information at the plurality of query node servers based on the mini-indexes and the full indexes;
- and
- publish the updated indexing information to one or more publishing channels in the distribution system based on a normal priority document updates, and publish the updated indexing information to the query nodes servers based on a higher priority document update.
A computer implemented method for rebuilding indexing information in near real time, the method comprising:
- generating, by a first indexing subsystem, full indexes to sending full indexes by the first indexing system by a plurality of query node servers via a distribution system, the first indexing system representing a map-reduced indexing system;
- generating, by a second indexing subsystem, mini-indexes associated with the full indexes;
- sending the mini-indexes, by the second indexing system, indirectly via the distribution system or directly to the plurality of query node servers to publish the document updates by the plurality of query node servers, the second indexing subsystem representing a daemon-based indexing system that monitors for and receives in near real-time event notifications, the event notifications based on a priority ordering of events by application servers;
- updating the indexing information at the plurality of query node servers based on the mini-indexes and the full indexes;
- and
- publishing the updated indexing information to one or more publishing channels in the distribution system based on a normal priority document update, and publish the updated indexing information to the query nodes servers based on a higher priority document update.
Dependencies are inferred from the claim text.