Dynamic outlier bias reduction system and method

A system for specialized for assessing the viability of a target data set for developing a mode for a financial instrument, comprising:

• an input unit configured to receive a target data set corresponding to a financial instrument, wherein the target data set comprises a plurality of data values for at least one target variable corresponding to the financial instrument;
• a computing unit coupled to the input unit, wherein the computing unit comprises a processor and a non-transient storage subsystem,
• a computer program stored by the non-transient storage subsystem comprising instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing a model to perform at least the following:
• generate a random data set based on the target data set;
• receive a plurality of bias criteria values used to determine one or more outliers;
• produce a plurality of outlier bias reduced target data sets that are associated with the bias criteria values by applying a mathematical model and a dynamic outlier bias reduction to the target data set;
• produce a plurality of outlier bias reduced random data sets that are associated with the bias criteria values by applying the mathematical model and the dynamic outlier bias reduction to the random data set;
• calculate at least one target error value for each of the outlier bias reduced target data sets and at least one random error value for each of the outlier bias reduced random data sets;
• calculate at least one target correlation value for each of the outlier bias reduced target data sets and at least one random correlation value for each of the outlier bias reduced random data sets;
• construct a first bias criteria curve for the target data set on a graph based on the at least one target error value and the at least one target correlation value for each of the outlier bias reduced target data sets;
• construct a second bias criteria curve for the random data set on the graph based on the at least one random error value and the at least one random correlation value for each of the outlier bias reduced random data sets;
• and
• compare the first bias criteria curve and the second bias criteria curve to determine viability of the target data set used for the mathematical model.

A system for reducing outlier bias in target variables measured for a facility, comprising:

• an input unit for inputting one or more data sets to be processed, wherein the input unit comprises a measuring device configured to:
• measure one or more target variables for the facility;
• and
• provide a corresponding data set for each of the target variables;
• a computing unit coupled to the input unit and for processing the one or more data sets, wherein the computing unit comprises a processor and a non-transient storage subsystem;
• an output unit coupled to the computing unit and for outputting one or more of the processed data sets received from the computing unit;
• and
• a computer program stored by the non-transient storage subsystem comprising instructions, when executed by the processor, cause the system specialized for reducing outlier bias in target variables measured for the facility to perform at least the following:
• receive at least one error threshold criteria and the corresponding data set via the database;
• perform a first iteration of outlier bias reduction for the corresponding data set that comprises:
• determining a set of predicted values by applying a model comprising at least one coefficient to the data set;
• comparing the set of predicted values to the data set to produce at least one set of error values;
• removing a plurality of data outliers from the data set determined from the at least one set of error values and the at least one error threshold criteria to generate an outlier filtered data set;
• and
• constructing an updated model comprising at least one updated coefficient from the outlier filtered data set;
• and
• perform a second iteration of outlier bias reduction for the data set based upon a determination that at least one termination criteria is not satisfied,
• wherein performing the second iteration of outlier bias reduction comprises determining a set of second predicted values by applying the updated model to the data set.