Anomaly detection for streaming data

A non-transitory computer-readable medium storing computer-executable instructions, that in response to execution by one or more hardware processors, causes the one or more hardware processors to perform operations comprising:

• receiving an input corresponding to one or more data pipeline metrics;
• generating a first plurality of data windows from a first portion of a plurality of streaming data, the first plurality of data windows corresponding to a first data pipeline metric of the one or more data pipeline metrics;
• generating a first plurality of data aggregations corresponding to the first plurality of data windows, wherein generating the first plurality of data aggregations includes aggregating data associated with each data window of the first plurality of data windows based on the first data pipeline metric;
• accessing a database to determine a first set of one or more reference data aggregations that corresponds to a first data aggregation of the first plurality of data aggregations based on a time interval and one or more additional temporal factors associated with the first data aggregation;
• and
• determining whether the first data aggregation corresponds to an anomaly, wherein the determining whether the first data aggregation corresponds to the anomaly is based on utilizing a model to compare the first set of one or more reference data aggregations to the first data aggregation.

A method, comprising:

• receiving, by a computer system, an input corresponding to a first data pipeline metric and a second data pipeline metric, wherein the first data pipeline metric is different than the second data pipeline metric;
• generating, by the computer system, a first plurality of data windows and a second plurality of data windows from a first portion of a plurality of streaming data, the first plurality of data windows corresponding to the first data pipeline metric and the second plurality of data windows corresponding to the second data pipeline metric;
• generating, by the computer system, a first plurality of data aggregations corresponding to the first plurality of data windows, and a second plurality of data aggregations corresponding to the second plurality of data windows, wherein generating the first plurality of data aggregations includes aggregating data associated with each data window of the first plurality of data windows based on the first data pipeline metric and generating the second plurality of data aggregations includes aggregating data associated with each window of the second plurality of data windows;
• accessing, by the computer system, a database to determine a first set of one or more reference data aggregations that corresponds to a first data aggregation of the first plurality of data aggregations and a second set of one or more reference data aggregations that corresponds to a second data aggregation of the second plurality of data aggregations, wherein the first data aggregation and the second data aggregation correspond to a first sub-portion of the first portion of the plurality of streaming data;
• and
• determining, by the computer system whether the first sub-portion corresponds to an anomaly based on determining if one or more of the first data aggregation or the second data aggregation corresponds to the anomaly or a different anomaly, wherein the determining whether the first data aggregation corresponds to the anomaly is based on utilizing a first anomaly model and determining whether the second data aggregation corresponds to the anomaly or a different anomaly is based on utilizing a second anomaly model.