Dataset cleansing

A computer system specifically configured for removing anomalies from a dataset comprising a plurality of ordered data elements, each of the plurality of ordered data elements comprising a sample of a system that operates according to a pattern, the system comprising:

• a memory operable to store the dataset;
• and
• a computer processor, coupled with the memory, the computer processor configured to:
• establish, using pattern logic stored in the memory, a historical pattern of values of the system and, based thereon, define, using smoothing logic stored in the memory, a threshold value;
• compare, using the smoothing logic, a value of each data element to the threshold value and generate a smoothed data set by removing any of the plurality of data elements from the data set when the value deviates from the threshold value;
• determine, using completion logic stored in the memory, subsequent to or concurrently with removal of any of the plurality of data elements, whether any data elements are missing from the smoothed data set and, for each data element determined to be missing, generate a new data element for the missing data element in accordance with the pattern;
• and
• generate, using the completion logic, a complete dataset comprising the data elements of the smoothed dataset and the new data elements.

A non-transitory computer readable medium including instructions for removing anomalies from a dataset comprising a plurality of ordered data elements stored in a memory, each of the plurality of ordered data elements comprising a sample of a system that operates according to a pattern, that when executed by the computer system are operable to:

• establish, using pattern logic stored in the memory, a pattern of values of the system and, based thereon, define, using smoothing logic stored in the memory, a threshold value;
• compare, using the smoothing logic, a value of each data element to the threshold value and generate a smoothed data set by removing any of the plurality of data elements from the data set when the value deviates from the threshold value;
• determine, using completion logic stored in the memory, subsequent to or concurrently with removal of any of the plurality of data elements, whether any data elements are missing from the smoothed data set and, for each data element determined to be missing, generate a new data element for the missing data element in accordance with the pattern;
• and
• generate, using the completion logic, a complete dataset comprising the data elements of the smoothed dataset and the new data elements.