Systems and methods for data quality control and cleansing
AssigneeJohnson Controls Technology Company
InventorsMichael J. Wenzel, Andrew J. Boettcher, Kirk H. Drees, James P. Kummer
Life of this patent
Estimating the term…
Abstract
A method for detecting and cleansing suspect building automation system data is shown and described. The method includes using processing electronics to automatically determine which of a plurality of error detectors and which of a plurality of data cleansers to use with building automation system data. The method further includes using processing electronics to automatically detect errors in the data and cleanse the data using a subset of the error detectors and a subset of the cleansers.
Claims20
A method for detecting and cleansing suspect building automation system data, comprising:
- using processing electronics to automatically determine which of a plurality of error detectors and which of a plurality of data cleansers to use with building automation system data;
- and
- using the processing electronics to automatically detect errors in the data and cleanse the data using a subset of the error detectors and a subset of the cleansers.
A computerized method for detecting suspect building automation system data using an adaptive bounds error detector, the method comprising:
- receiving a critical value and a data window having a window size;
- collecting data points until the number of data points collected equals the window size;
- using robust statistics to estimate data spread and central tendency in the current data window;
- collecting a next data point having a data value;
- and
- flagging the next data point as suspect data if the absolute value of the difference between the data value and the central tendency is greater than the product of the critical value and the data spread.
A computerized method for detecting suspect building automation system data using an adaptive derivative bounds detector, the method comprising:
- receiving a critical derivative and a growth function;
- collecting an initial data point having an initial data value and an initial data timestamp and a next data point having a next data value and a next data timestamp;
- determining the rate of change in data value between the data points by dividing the difference in data values by the difference in data timestamps;
- flagging the next data point as suspect data if the absolute value of the rate of change in data value between the data points is greater than the critical derivative;
- and
- using the growth function to either
- (a) increase the critical derivative if the absolute value of the rate of change in data value between the data points is greater than the critical derivative, or
- (b) decrease the critical derivative if the absolute value of the rate of change in data value between the data points is less than or equal to the critical derivative.
A computerized method for detecting suspect building automation system data using a stuck data detector, the method comprising:
- receiving a maximum false alarm rate and a plurality of data points, each data point having a data value and a data timestamp;
- determining an average time between changes in data value;
- determining a first critical time based on the average time between changes and the maximum false alarm rate;
- receiving a next data point having a next data value and a next data timestamp;
- and
- flagging the next data point as stuck if the difference between the next data timestamp and the timestamp of the most recent previous data point with a data value different from the next data value is greater than the first critical time.
Dependencies are inferred from the claim text.