Framework for failure detection, forecasting and remediation combining domain-specific telemetry and customized machine learning models

A system comprising:

• a memory configured to store program instructions for a framework for forecasting potential component failures of at least one information technology (IT) system, the framework including a queuing system, a telemetry processing component, a feature engineering component, a machine learning (ML) component, and an inferencing microservice;
• and
• processing circuitry configured to execute the program instructions out of the memory to:
• collect, in real-time at the queuing system, telemetry data specific to multiple different IT system component domains of the at least one IT system;
• place, in separate data queues of the queuing system, the telemetry data specific to the multiple different IT system component domains of the at least one IT system;
• for each separate data queue containing telemetry data specific to a respective IT system component domain from among the multiple different IT system component domains:
• extract, by the telemetry processing component, the telemetry data from the separate data queue;
• generate, by the feature engineering component, feature data based on the extracted telemetry data specific to the respective IT system component domain;
• forecast, by the inferencing microservice, a potential failure of a component in the respective IT system component domain based on an output of a customized ML model, the customized ML model operating on first obtained feature data for the respective IT system component domain, and the customized ML model being trained using the generated feature data for the respective IT system component domain;
• and
• in response to forecasting the potential failure of the component in the respective IT system component domain, perform a remedial action,
• wherein collecting the telemetry data specific to the multiple different IT system component domains of the at least one IT system includes collecting, in real-time at the queuing system, a greater amount of first telemetry data specific to a first IT system component domain in a first data queue of the queuing system, and collecting, in real-time at the queuing system, a lesser amount of second telemetry data specific to a second IT system component domain in a second data queue of the queuing system, and
• wherein forecasting the potential failure of the component in the respective IT system component domain includes forecasting, by a first instance of the inferencing microservice, a potential failure of a first component in the first IT system component domain, and forecasting, by a second instance of the inferencing microservice, a potential failure of a second component in the second IT system component domain;
• automatically scale-up the first instance of the inferencing microservice for forecasting the potential failure of the first component in the first IT system component domain based on the greater amount of the first telemetry data in the first data queue;
• and
• automatically scale-down the second instance of the inferencing microservice for forecasting the potential failure of the second component in the second IT system component domain.

A computer program product including a set of non-transitory, computer-readable media having program instructions that, when executed by processing circuitry, cause the processing circuitry to perform a method comprising:

• providing a computer-executable framework for forecasting potential component failures of at least one information technology (IT) system, the computer-executable framework including a queuing system, a telemetry processing component, a feature engineering component, a machine learning (ML) component, and an inferencing microservice;
• collecting, in real-time at the queuing system, telemetry data specific to multiple different IT system component domains of the at least one IT system;
• placing, in separate data queues of the queuing system, the telemetry data specific to the multiple different IT system component domains of the at least one IT system;
• for each separate data queue containing telemetry data specific to a respective IT system component domain from among the multiple different IT system component domains:
• extracting, by the telemetry processing component, the telemetry data from the separate data queue;
• generating, by the feature engineering component, feature data based on the extracted telemetry data specific to the respective IT system component domain;
• forecasting, by the inferencing microservice, a potential failure of a component in the respective IT system component domain based on an output of a customized ML model, the customized ML model operating on first obtained feature data for the respective IT system component domain, and the customized ML model being trained using the generated feature data for the respective IT system component domain;
• and
• in response to forecasting the potential failure of the component in the respective IT system component domain, performing a remedial action,
• wherein the collecting of the telemetry data specific to the multiple different IT system component domains of the at least one IT system includes collecting, in real-time at the queuing system, a greater amount of first telemetry data specific to a first IT system component domain, and collecting, in real-time at the queuing system, a lesser amount of second telemetry data specific to a second IT system component domain, and
• wherein the forecasting of the potential failure of the component in the respective IT system component domain includes forecasting, by a first instance of the inferencing microservice, a potential failure of a first component in the first IT system component domain, and forecasting, by a second instance of the inferencing microservice, a potential failure of a second component in the second IT system component domain;
• automatically scaling-up the first instance of the inferencing microservice for forecasting the potential failure of the first component in the first IT system component domain based on the greater amount of the first telemetry data;
• and
• automatically scaling-down the second instance of the inferencing microservice for forecasting the potential failure of the second component in the second IT system component domain based on the lesser amount of the second telemetry data.