US 2015/0032707 A1 · SYSTEMS AND METHODS FOR PRUNING DATA BY SAMPLING

Abstract

Techniques provided herein allow for management of data. In various embodiments, systems and methods prune and retain data being managed by a data management system, where the managed data can include log data aggregated from one or more servers for analysis purposes. According to some embodiments, pruning can be triggered according to one or more constraints, such as the age of managed data (e.g., retain only 30 days of managed data) or the memory space required to store the managed data (e.g., retain only 100 GB worth of managed data). The constraints that trigger data pruning can be based on a data retention policy. When triggered, pruning can be performed on a fraction of the managed data stored based on the data retention policy (e.g., 3 days of full managed data, 27 days of pruned managed data). The pruning may be performed by sampling, at a desired rate, the managed data.

Claims20

Dependent claims

A computer system comprising:

at least one processor;
and
a memory storing instructions configured to instruct the at least one processor to perform:
detecting when a constraint for storing a data set has been exceeded;
identifying, based on the constraint, an initial data subset from the data set from which at least some data elements will be removed by sampling;
determining a sampling rate for data element retention in the initial data subset;
identifying a secondary data subset from the initial data subset based on sampling the initial data subset according to the sampling rate;
and
removing from the data set one or more data elements of the initial data subset while retaining data elements of the secondary data subset.

6 more elements

Dependencies are inferred from the claim text.

Systems and methods for pruning data by sampling

Life of this patent

Systems and methods for pruning data by sampling

Life of this patent

Abstract

Claims20