Evicting data associated with a data intake and query system from a local storage
AssigneeSplunk Inc.
InventorsBharath Aleti, Alexandros Batsakis, Paul J. Lucas, Igor Stojanovski
Life of this patent
Estimating the term…
Abstract
Systems and methods are disclosed for making space available in a local storage of a data intake and query system. A cache manager of the data intake and query system may determine an amount of storage space of a local data store that is available for use to perform a query. The cache manager may then use one or more eviction policies associated with content stored at the local data store to purge content items to evict from the local storage. The system may then retrieve content for performing the query from a remote storage and store the retrieved content at the local storage.
Claims20
A method, comprising:
- receiving, by a data intake and query system, a query identifying a set of data to be processed and a manner of processing the set of data;
- identifying content of a set of buckets for retrieval from a remote data store based on the query;
- determining, based at least in part on the content of the set of buckets for retrieval, that storage available on a local data store does not satisfy an available storage threshold;
- identifying, from a plurality of eviction policies, a first eviction policy associated with a first bucket of a plurality of buckets stored on the local data store, wherein the first eviction policy associated with the first bucket comprises a first eviction criterion used to evaluate a first portion of content of the first bucket for potential eviction;
- evicting the first portion of the content of the first bucket based on the first eviction policy;
- determining that a second bucket of the plurality of buckets satisfies at least one query parameter of the query;
- and
- identifying the content of the second bucket for retention on the local data store based at least in part on said determining that the second bucket satisfies at least one query parameter of the query.
A computing system of a data intake and query system, the computing system comprising:
- memory;
- and
- one or more processing devices coupled to the memory and configured to:
- receive a query identifying a set of data to be processed and a manner of processing the set of data;
- identify content of a set of buckets for retrieval from a remote data store based on the query;
- determine, based at least in part on the content of the set of buckets for retrieval, that storage available on a local data store does not satisfy an available storage threshold;
- identify, from a plurality of eviction policies, a first eviction policy associated with a first bucket of a plurality of buckets stored on the local data store, wherein the first eviction policy associated with the first bucket comprises a first eviction criterion used to evaluate a first portion of content of the first bucket for potential eviction;
- evict the first portion of the content of the first bucket based on the first eviction policy;
- determine that a second bucket of the plurality of buckets satisfies at least one query parameter of the query;
- and
- identify the content of the second bucket for retention on the local data store based at least in part on said determining that the second bucket satisfies at least one query parameter of the query.
Non-transitory computer readable media comprising computer-executable instructions that, when executed by a computing system of a data intake and query system, cause the computing system to:
- receive a query identifying a set of data to be processed and a manner of processing the set of data;
- identify content of a set of buckets for retrieval from a remote data store based on the query;
- determine, based at least in part on the content of the set of buckets for retrieval, that storage available on a local data store does not satisfy an available storage threshold;
- identify, from a plurality of eviction policies, a first eviction policy associated with a first bucket of a plurality of buckets stored on the local data store, wherein the first eviction policy associated with the first bucket comprises a first eviction criterion used to evaluate a first portion of content of the first bucket for potential eviction;
- evict the first portion of the content of the first bucket based on the first eviction policy;
- determine that a second bucket of the plurality of buckets satisfies at least one query parameter of the query;
- and
- identify the content of the second bucket for retention on the local data store based at least in part on said determining that the second bucket satisfies at least one query parameter of the query.
Dependencies are inferred from the claim text.