Improving data monitoring and quality using AI and machine learning
AssigneeHewlett Packard Enterprise Development LP
InventorsTahir Cader, Sergey Serebryakov, Torsten Wilde, Jeff Hanson
Life of this patent
Estimating the term…
Abstract
Systems and methods are provided for improving statistical and machine learning drift detection models that monitor computing health of a data center environment. For example, the system can receive streams of sensor data from a plurality of sensors in a data center; clean the streams of sensor data; generate, using a machine learning (ML) model, an anomaly score and a dynamic threshold value based on the cleaned streams of sensor data; determine, using the ML model and based on the anomaly score and the dynamic threshold value, a correctness indicator for a first sensor in the plurality of sensors; and using the correctness indicator, correct the first sensor.
Claims21
A computing device comprising:
- a non-transitory memory;
- and
- one or more processors that are configured to execute machine readable instructions stored in the non-transitory memory for causing the one or more processors to:
- receive streams of sensor data from a plurality of sensors in a data center;
- for an incoming stream of sensor data from the streams of sensor data, determine that a first sensor in the plurality of sensors that generated the incoming stream of sensor data has been recalibrated since a previous stream was received;
- initiate a correction process of the incoming stream of sensor data to generate a cleaned stream of sensor data;
- generate, using a machine learning (ML) model, an anomaly score and a dynamic threshold value, the dynamic threshold value being compared to the cleaned stream of sensor data in detecting anomalous data generated by the plurality of sensors, and the anomaly score comprising data that indicates characteristics of the plurality of sensors or measured attributes of the data center;
- determine, using the ML model and based on the anomaly score and the dynamic threshold value, a correctness indicator for a first sensor in the plurality of sensors;
- identifying seasonality changes in temperature data of the data center;
- adjusting, using the ML model, the dynamic threshold value associated with the seasonality changes, wherein adjusting the dynamic threshold value also adjusts the correctness indicator for the first sensor in the plurality of sensors;
- and
- using the correctness indicator, correct the first sensor.
A computer-implemented method comprising:
- receiving streams of sensor data from a plurality of sensors in a data center;
- for an incoming stream of sensor data from the streams of sensor data, determining that a first sensor in the plurality of sensors that generated the incoming stream of sensor data has been recalibrated since a previous stream was received;
- initiating a correction process of the incoming stream of sensor data to generate a cleaned stream of sensor data;
- generating, using a machine learning (ML) model, an anomaly score and a dynamic threshold value, the dynamic threshold value being compared to the cleaned stream of sensor data in detecting anomalous data generated by the plurality of sensors, and the anomaly score comprising data that indicates characteristics of the plurality of sensors or measured attributes of the data center;
- determining, using the ML model and based on the anomaly score and the dynamic threshold value, a correctness indicator for a first sensor in the plurality of sensors;
- identifying seasonality changes in temperature data of the data center;
- adjusting, using the ML model, the dynamic threshold value associated with the seasonality changes, wherein adjusting the dynamic threshold value also adjusts the correctness indicator for the first sensor in the plurality of sensors;
- and
- using the correctness indicator, correct the first sensor.
A non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, the plurality of instructions when executed by the one or more processors cause the one or more processors to:
- receive streams of sensor data from a plurality of sensors in a data center;
- for an incoming stream of sensor data from the streams of sensor data, determine that a first sensor in the plurality of sensors that generated the incoming stream of sensor data has been recalibrated since a previous stream was received;
- initiate a correction process of the incoming stream of sensor data to generate a cleaned the streams of sensor data;
- generate, using a machine learning (ML) model, an anomaly score and a dynamic threshold value, the dynamic threshold value being compared to the cleaned stream of sensor data in detecting anomalous data generated by the plurality of sensors, and the anomaly score comprising data that indicates characteristics of the plurality of sensors or measured attributes of the data center;
- determine, using the ML model and based on the anomaly score and the dynamic threshold value, a correctness indicator for a first sensor in the plurality of sensors;
- identify seasonality changes in temperature data of the data center;
- adjust, using the ML model, the dynamic threshold value associated with the seasonality changes, wherein adjusting the dynamic threshold value also adjusts the correctness indicator for the first sensor in the plurality of sensors;
- and
- using the correctness indicator, correct the first sensor.
Dependencies are inferred from the claim text.