Integrated search for shared storage using index throttling to maintain quality of service

A method of searching shared files stored on a shared storage comprising:

• providing a search functionality in the shared storage, the search functionality having a set of search indexes to perform a search based on a search term;
• providing an event stream of update events associated with the shared files in the shared storage, wherein the update events comprise file update events, which include file modification and file creation, and file deletion events;
• processing the event stream by performing event filtering, event compression and de-duplication on the stream of update events to generate a processed event stream;
• and
• responding to update events in the processed event stream by indexing files referenced in the update events in the processed event stream, the indexing being performed to maintain a predetermined level of quality of service at the shared storage,
• wherein processing the event stream to generate a processed event stream comprises:
• processing the event stream using an event queue;
• identifying each update event using a file identification uniquely identifying a file in the shared storage;
• receiving a first update event having a first file identification;
• and
• generating the processed event stream by removing a duplicate update event having the same file identification as the first update event.

A system for searching shared files stored on a shared storage comprising:

• a processor;
• and
• a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
• provide a search functionality in the shared storage, the search functionality having a set of search indexes to perform a search based on a search term;
• provide an event stream of update events associated with the shared files in the shared storage, wherein the update events comprise file update events, which include file modification and file creation, and file deletion events;
• process the event stream by performing event filtering, event compression and de-duplication on the stream of update events to generate a processed event stream;
• and
• respond to update events in the processed event stream by throttling the indexing of files referenced in the update events in the processed event stream to maintain a predetermined level of quality of service at the shared storage.