Reassigning processing tasks to an external storage system
AssigneeSplunk Inc.
InventorsSourav Pal, Arindam Bhattacharjee, Wayne Patterson
Life of this patent
Estimating the term…
Abstract
Systems and methods are described for reducing execution time of a query that references external data systems. The system can determine an external data system is capable of processing one or more map or reduce phases of a map-reduce operation. When it is determined that the external data system can process a map or reduce phase, associated operations may be reassigned from the system to the external data system reducing the processing resources used by the system to response to the query and, in some cases, speeding up performance of the query.
Claims30
A method, comprising:
- receiving a query that references an external search for a remote data store;
- determining that the remote data store includes processing capabilities;
- determining a type of processing capabilities of the remote data store;
- identifying additional processing to be performed on results of the external search based on the query;
- generating additional processing instructions based on the identified additional processing to be performed on the results of the external search and the determined type of processing capabilities of the remote data store;
- and
- communicating the external search in combination with the additional processing instructions.
A computing system of a data intake and query system, the computing system comprising:
- memory;
- and
- one or more processing devices coupled to the memory and configured to:
- receive a query that references an external search for a remote data store;
- determine that the remote data store includes processing capabilities;
- determine a type of processing capabilities of the remote data store;
- identify additional processing to be performed on results of the external search based on the query;
- generate additional processing instructions based on the identified additional processing to be performed on the results of the external search and the determined type of processing capabilities of the remote data store;
- and
- communicate the external search in combination with the additional processing instructions to the remote data store or a worker node configured to interact with the remote data store.
Non-transitory computer-readable media comprising computer-executable instructions that, when executed by a computing system of a data intake and query system, cause the computing system to:
- receive a query that references an external search for a remote data store;
- determine that the remote data store includes processing capabilities;
- determine a type of processing capabilities of the remote data store;
- identify additional processing to be performed on results of the external search based on the query;
- generate additional processing instructions based on the identified additional processing to be performed on the results of the external search and the determined type of processing capabilities of the remote data store;
- and
- communicate the external search in combination with the additional processing instructions to the remote data store or a worker node configured to interact with the remote data store.
Dependencies are inferred from the claim text.