Record profiling for dataset sampling

A system for generating a smaller dataset from a larger dataset, each dataset holding a plurality of records, the system comprising:

• means for profiling the larger dataset to identify a plurality of patterns, where each of the plurality of patterns is descriptive of one or more records of the plurality of records held in the larger dataset;
• means for filling the smaller dataset with records held in the larger dataset, where the smaller dataset has a predetermined size in bytes;
• means for individually retrieving multiple records held in the larger dataset;
• and
• for each retrieved record, means for determining whether to (a) place the retrieved record into the smaller dataset and, if placing the retrieved record into the smaller dataset would exceed the predetermined size, evict a record already in the smaller dataset;
• or (b) not place the retrieved record into the smaller dataset;
• where the above-cited determining is based on a pattern of the retrieved record and a representation status of the pattern in the smaller dataset.

A method for generating a smaller dataset from a larger dataset, each dataset holding a plurality of alphanumeric strings, the method comprising:

• profiling the larger dataset to identify a plurality of patterns, each of which is descriptive of one or more alphanumeric strings of the plurality of alphanumeric strings held in the larger dataset;
• filling each of a plurality of slots of the smaller dataset with alphanumeric strings held in the larger dataset;
• individually retrieving each alphanumeric string held in the larger dataset;
• and
• for each retrieved alphanumeric string, determining whether to (a) place the retrieved alphanumeric string into a slot of the smaller dataset and evict an alphanumeric string already occupying that slot;
• or (b) not place the retrieved alphanumeric string into the smaller dataset;
• where the above-cited determining is based on a pattern of the retrieved alphanumeric string and a representation status of the pattern in the smaller dataset, and the above-cited determining is done so as to increase a number of unique patterns represented in the smaller dataset.