Learning from mistakes to improve detection rates of machine learning (ML) models
AssigneeZscaler, Inc.
InventorsDianhuan Lin, Miao Zhang, Shaleen Taneja, Rex Shang, Howie Xu
Life of this patent
Estimating the term…
Abstract
Systems and methods for learning from mistakes to improve detection rates of Machine Learning (ML) models. The systems and methods including receiving data with labels; running the data through a trained ML model for predictions; identifying errors in the predictions based on the labels received with the data; adjusting weights associated with samples in the data based on the identified errors; and retraining the ML model with the adjusted weights.
Claims20
A method for improving precision and recall (detection rates) of Machine Learning (ML) models for malware detection in a computing network environment, the method comprising steps of:
- receiving data with labels identifying files, network flows, or executables as malicious or benign;
- running the data through a trained ML model for predictions;
- identifying errors in the predictions based on the labels received with the data, wherein the errors including false positives and false negatives;
- performing error analysis to group error-causing samples by root-cause feature sets;
- adjusting weights associated with samples in the data based on the identified errors, wherein adjusting comprises adding weight increments to error-causing samples and combining those increments with pre-existing fixed sample weights;
- and
- retraining the ML model with the adjusted weights to improve subsequent malware detection performance in the cloud-based system.
A cloud-based system for improving detection rates of Machine Learning (ML) models used in a computing network environment, the cloud based system comprising:
- one or more processors and memory storing instructions that, when executed, cause the one or more processors to:
- receive data with labels identifying files, network flows, or executables as malicious or benign;
- run the data through a trained ML model for predictions;
- identify errors in the predictions based on the labels received with the data, wherein the errors including false positives and false negatives;
- perform error analysis to group error-causing samples root-cause feature sets;
- adjust weights associated with samples in the data based on the identified errors, wherein the weights are adjusted by adding weight increments to error-causing samples and combining those increments with pre-existing fixed sample weights;
- and
- retrain the ML model with the adjusted weights to improve subsequent malware detection performance in the cloud-based system.
A non-transitory computer-readable medium comprising instructions for improving detection rates of Machine Learning (ML) models for malware detection in a computing network environment that, when executed, cause one or more processors to perform steps of:
- receiving data with labels identifying files, network flows, or executables as malicious or benign;
- running the data through a trained ML model for predictions;
- identifying errors in the predictions based on the labels received with the data, wherein the errors including false positives and false negatives;
- performing error analysis to group error-causing samples by root-cause feature sets;
- adjusting weights associated with samples in the data based on the identified errors, wherein adjusting comprises adding weight increments to error-causing samples and combining those increments with pre-existing fixed sample weights;
- and
- retraining the ML model with the adjusted weights to improve subsequent malware detection performance in the cloud-based system.
Dependencies are inferred from the claim text.