US 2024/0205100 A1 · AUTOMATED ANOMALY DETECTION MODEL QUALITY ASSURANCE AND DEPLOYMENT FOR WIRELESS NETWORK FAILURE DETECTION

Abstract

Systems and methods are provided for automated anomaly detection model quality assurance (QA) and deployment for wireless network failure prediction. Network failure prediction can leverage models trained to detect issues on a network and predict failure scenarios by identifying anomalous issues indicative of failure conditions. To keep these models up to date with changes in network behavior and configurations, the models are recalibrated from time to time. Implementations disclosed herein provide for automated evaluation and deployment of recalibrated models, while assuring issue detection results from the recalibrated models accurately reflect current network conditions. To do this, implementations disclosed herein determine QA metrics for recalibrated, candidate models, QA thresholds from previously deployed models, and QA criteria from a currently deployed model. Based on a comparison of the QA metrics with the QA thresholds and the QA criteria, implementations disclosed herein automatically deploy recalibrated, candidate models without human or external intervention.

Claims20

Dependent claims

A method for automated model quality assurance, the method comprising:

receiving network failure prediction data associated with a candidate model based on applying testing data indicative of current network operating conditions;
calculating a plurality of quality assurance metrics for the candidate model from the received prediction data;
determining a plurality of thresholds based on a plurality of performance distributions derived from a plurality of previously deployed models corresponding to the candidate model, the plurality of performance distributions based on applying the testing data to each of the plurality of previously deployed models;
determining a criteria based on a prediction from a currently deployed model corresponding to the candidate model based on applying the testing data to the currently deployed model;
and
automatically deploying the candidate model based on a comparison of the plurality of quality assurance metrics with the plurality of thresholds and a comparison of the plurality of quality assurance metrics with the criteria,
wherein each of the currently deployed model, the candidate model, and the plurality of previously deployed models are configured to predict failure scenarios of a network by detecting anomalous issues occurring on the network from real-time data from network functions.

4 more elements

A system for automated model quality assurance, comprising:

at least one memory configured to store instructions;
and
one or more processors communicably coupled to the memory and configured to execute the instruction to:
receive network failure prediction data associated with a candidate model based on applying testing data indicative of current network operating conditions;
calculate a plurality of quality assurance metrics for the candidate model from the received prediction data;
determine a plurality of thresholds based on a plurality of performance distributions derived from a plurality of previously deployed models corresponding to the candidate model, the plurality of performance distributions based on applying the testing data to each of the plurality of previously deployed models;
determine a criteria based on a prediction from a currently deployed model corresponding to the candidate model based on applying the testing data to the currently deployed model;
and
automatically deploy the candidate model based on a comparison of the plurality of quality assurance metrics with the plurality of thresholds and a comparison of the plurality of quality assurance metrics with the criteria,
wherein each of the currently deployed model, the candidate model, and the plurality of previously deployed models are configured to predict failure scenarios of a network by detecting anomalous issues occurring on the network from real-time data from network functions.

7 more elements

Dependencies are inferred from the claim text.

Automated anomaly detection model quality assurance and deployment for wireless network failure detection

Life of this patent

Automated anomaly detection model quality assurance and deployment for wireless network failure detection

Life of this patent

Abstract

Claims20