Workload pool hierarchy for a search and indexing system
AssigneeSplunk Inc.
InventorsBharath Kishore Reddy Aleti, Alexandros Batsakis, Mitchell Neuman Blank, Rama Gopalan, Hongxun Liu, Anish Shrigondekar
Life of this patent
Estimating the term…
Abstract
Resource management includes storing, for multiple workload pools of a data intake and query system, a workload pool hierarchy arranged in multiple workload pool layers. After storing a processing request is assigned a selected subset of workload pools in a second layer of the workload pool hierarchy based on a type of processing request. The processing request is then assigned to an individual workload pool in the selected subset to obtain a selected workload pool. Execution of the processing request is initiated on the selected workload pool.
Claims20
A method comprising:
- storing, for a plurality of workload pools of a data intake and query system, a workload pool hierarchy arranged in a plurality of workload pool layers comprising:
- a first layer comprising a node for allocating a set of workload pools of the plurality of workload pools to data intake and query system processes,
- a second layer partitioning the set of workload pools for data intake and query system processes into a first subset allocated for query processes, a second subset allocated for ingest processes, and a third subset allocated for user input processes,
- wherein the data intake and query system processes comprise the query processes, the ingest processes, and the user input processes,
- wherein the first subset, the second subset, and the third subset are disjoint, and
- wherein the ingest processes comprise processes to ingest data into the data intake and query system, the query processes comprise processes to query the data in the data intake and query system, and the user input processes comprise processes to monitor and manage the data intake and query system, and
- a third layer partitioning the first subset, the second subset, and the third subset into individual workload pools, wherein the individual workload pools are in leaf nodes of the workload pool hierarchy;
- assigning a processing request to a subset selected from a group consisting of the first subset, the second subset, and the third subset based on a type of processing request, wherein the type of processing request is for one selected from a group consisting of the query processes, the ingest processes, and the user input processes;
- assigning the processing request to an individual workload pool in the selected subset to obtain a selected workload pool;
- and
- initiating execution of the processing request on the selected workload pool,
- wherein each of the plurality of workload pools is a non-overlapping partition of hardware resources as defined by the workload pool hierarchy.
A system comprising:
- memory comprising instructions;
- and
- a computer processor for executing the instructions that cause the computer processor to perform operations comprising:
- storing, for a plurality of workload pools of a data intake and query system, a workload pool hierarchy arranged in a plurality of workload pool layers comprising:
- a first layer comprising a node for allocating a set of workload pools of the plurality of workload pools to data intake and query system processes,
- a second layer partitioning the set of workload pools for data intake and query system processes into a first subset allocated for query processes, a second subset allocated for ingest processes, and a third subset allocated for user input processes, wherein the data intake and query system processes comprise the query processes, the ingest processes, and the user input processes, wherein the first subset, the second subset, and the third subset are disjoint, and
- wherein the ingest processes comprise processes to ingest data into the data intake and query system, the query processes comprise processes to query the data in the data intake and query system, and the user input processes comprise processes to monitor and manage the data intake and query system, and
- a third layer partitioning the first subset, the second subset, and the third subset into individual workload pools, wherein the individual workload pools are in leaf nodes of the workload pool hierarchy,
- assigning a processing request to a subset selected from a group consisting of the first subset, the second subset, and the third subset based on a type of processing request, wherein the type of processing request is for one selected from a group consisting of the query processes, the ingest processes, and the user input processes,
- assigning the processing request to an individual workload pool in the selected subset to obtain a selected workload pool, and
- initiating execution of the processing request on the selected workload pool,
- wherein each of the plurality of workload pools is a non-overlapping partition of hardware resources as defined by the workload pool hierarchy.
A non-transitory computer readable medium comprising computer readable program code for performing operations, the operations comprising:
- storing, for a plurality of workload pools of a data intake and query system, a workload pool hierarchy arranged in a plurality of workload pool layers comprising:
- a first layer comprising a node for allocating a set of workload pools of the plurality of workload pools to data intake and query system processes,
- a second layer partitioning the set of workload pools for data intake and query system processes into a first subset allocated for query processes, a second subset allocated for ingest processes, and a third subset allocated for user input processes, wherein the data intake and query system processes comprise the query processes, the ingest processes, and the user input processes, wherein the first subset, the second subset, and the third subset are disjoint, and wherein the ingest processes comprise processes to ingest data into the data intake and query system, the query processes comprise processes to query the data in the data intake and query system, and the user input processes comprise processes to monitor and manage the data intake and query system, and
- a third layer partitioning the first subset, the second subset, and the third subset into individual workload pools, wherein the individual workload pools are in leaf nodes of the workload pool hierarchy;
- assigning a processing request to a subset selected from a group consisting of the first subset, the second subset, and the third subset based on a type of processing request, wherein the type of processing request is for one selected from a group consisting of the query processes, the ingest processes, and the user input processes;
- assigning the processing request to an individual workload pool in the selected subset to obtain a selected workload pool;
- and
- initiating execution of the processing request on the selected workload pool,
- wherein each of the plurality of workload pools is a non-overlapping partition of hardware resources as defined by the workload pool hierarchy.
Dependencies are inferred from the claim text.