# US 2022/0044129 A1 · Alert management in data processing systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044129 A1
- Document ID: US20220044129A1
- Status: Published application
- Kind: A1
- Application number: 16/986,342
- Filed: 2020-08-06
- Published: 2022-02-10
- Earliest claimed: 2020-08-06
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Michael Elton Nidd; Hagen Völzer; Ioana Giurgiu; Jinho Hwang; Larisa Shwartz
- Web page: https://patentgrep.com/patent/US20220044129A1

## CPC

- G06N 5/04: Inference or reasoning models
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/20: Ensemble learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Several aspects are provided for dynamically updating an alert-management system that uses a master ruleset to match alerts in a data processing system with automata for handling the alerts. A method comprises training a machine learning model to correlate the alerts with the automata using a training dataset comprising alerts which were successfully handled by the automata. The machine learning model is then applied to correlate unmatched alerts with the automata, wherein the unmatched alerts were not matched to the automata by the master ruleset. The method further comprises analyzing operation of the machine learning model in relation to correlation of the unmatched alerts to define a new ruleset for matching the unmatched alerts with the automata and outputting the new ruleset for auditing of each rule in the new ruleset. In response to approval of an audited rule, the audited rule is added to the master ruleset.

## Claims

### Claim 1

A computer-implemented method for dynamically updating an alert management system that uses a master ruleset to match alerts in a data processing system with automated processes (“automata”) for handling the alerts, the method comprising:
training, by one or more processors, a machine learning model to correlate the alerts with the automata using a training dataset comprising matched alerts successfully handled by the automata;
applying, by the one or more processors, the machine learning model to correlate unmatched alerts with the automata, wherein the unmatched alerts were not matched to the automata by the master ruleset;
analyzing, by the one or more processors, operation of the machine learning model in relation to correlation of the unmatched alerts to define a new ruleset for matching the unmatched alerts with the automata;
outputting, by the one or more processors, the new ruleset for auditing of each rule in the new ruleset; and
responsive to approval of an audited rule of the new ruleset, adding, by the one or more processors, the audited rule to the master ruleset.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein:
the machine learning model comprises a tree-ensemble model; and
the step of analyzing operation of the machine learning model further comprises:
generating, by the one or more processors, a new tree based on features of paths through trees in the tree-ensemble model via which the unmatched alerts were correlated with the automata, wherein the new tree defines the new ruleset.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further comprising:
pruning, by the one or more processors, the paths based on a confidence value calculated for each path, wherein the confidence value is an estimation of a correlation accuracy; and
generating, by the one or more processors, the new tree based on features of unpruned paths.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, further comprising:
pruning, by the one or more processors, the paths based on a number of the unmatched alerts correlated with the automata via respective paths; and
generating, by the one or more processors, the new tree based on features of unpruned paths.

### Claim 5 (depends on 2)

The computer-implemented method of claim 2, further comprising:
pruning, by the one or more processors, the paths based on a confidence value calculated for each path and on a number of the unmatched alerts correlated with the automata via respective paths, wherein the confidence value is an estimation of a correlation accuracy; and
generating, by the one or more processors, the new tree based on features of unpruned paths.

### Claim 6 (depends on 2)

The computer-implemented method of claim 2, wherein the step of analyzing operation of the machine learning model further comprises:
selecting, by the one or more processors, features from the trees of the tree-ensemble model based on significance values, each significance value indicating importance of the associated feature for correlating alerts in the training dataset with one of the automata with which a respective unmatched alert was correlated; and
generating, by the one or more processors, the new tree based on the selected features.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, further comprising:
pruning, by the one or more processors, the selected features based on a number of the unmatched alerts correlated with the automata via the paths including each feature; and
generating, by the one or more processors, the new tree based on unpruned features.

### Claim 8 (depends on 6)

The computer-implemented method of claim 6, further comprising:
pruning, by the one or more processors, the selected features based on confidence values calculated for respective paths via which the unmatched alerts were correlated with the automata and which included a respective selected feature of the selected features, wherein the confidence value is an estimation of a correlation accuracy; and
generating, by the one or more processors, the new tree based on unpruned features.

### Claim 9 (depends on 6)

The computer-implemented method of claim 6, further comprising:
pruning, by the one or more processors, the selected features based on a number of the unmatched alerts correlated with the automata via the paths including each feature, and based on confidence values calculated for the respective paths, wherein the confidence value is an estimation of a correlation accuracy; and
generating, by the one or more processors, the new tree based on unpruned features.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the step of analyzing operation of the machine learning model further comprises:
responsive to applying the machine learning model to obtain a correlation result for an unmatched alert, using, by the one or more processors, a model-independent explanation technique to identify at least one element of the alert as an important element for the correlation result; and
generating, by the one or more processors, the new ruleset based on the at least one element of the unmatched alerts.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein the step of analyzing operation of the machine learning model further comprises:
when applying the machine learning model to correlate each unmatched alert, calculating, by the one or more processors, an initial confidence value for the initial correlation result, wherein the initial confidence value is an estimation of a correlation accuracy;
for different elements of the alert, removing, by the one or more processors, that element and reapplying the machine learning model to the resulting alert to calculate a new confidence value for the initial correlation result;
identifying, by the one or more processors, at least one element of the alert as an important element for the initial correlation result based on a difference between the initial confidence value and the new confidence value for the different elements; and
generating, by the one or more processors, the new ruleset based on the at least one element of the unmatched alerts.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, wherein the machine learning model comprises a boosted random forests model.

### Claim 13

A computer program product for dynamically updating an alert-management system that uses a master ruleset to match alerts in a data processing system with automated processes (“automata”) for handling the alerts, the computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising:
program instructions to train a machine learning model to correlate the alerts with the automata using a training dataset comprising matched alerts successfully handled by the automata;
program instructions to apply the machine learning model to correlate unmatched alerts with the automata, wherein the unmatched alerts were not matched to the automata by the master ruleset;
program instructions to analyze operation of the machine learning model in relation to correlation of the unmatched alerts to define a new ruleset for matching the unmatched alerts with the automata;
program instructions to output the new ruleset for auditing of each rule in the new ruleset; and
responsive to approval of an audited rule of the new ruleset, program instructions to add the audited rule to the master ruleset.

### Claim 14 (depends on 13)

The computer program product of claim 13, wherein:
the machine learning model comprises a tree-ensemble model; and
the program instructions to analyze operation of the machine learning model further comprise:
program instructions to generate a new tree based on features of paths through trees in the tree-ensemble model via which the unmatched alerts were correlated with the automata, wherein the new tree defines the new ruleset.

### Claim 15 (depends on 14)

The computer program product of claim 14, further comprising:
program instructions to prune the paths based on a confidence value calculated for each path, wherein the confidence value is an estimation of a correlation accuracy; and
program instructions to generate the new tree based on features of unpruned paths.

### Claim 16 (depends on 14)

The computer program product of claim 14, further comprising:
program instructions to select features from the trees of the tree-ensemble model based on significance values, each significance value indicating importance of the associated feature for correlating the alerts in the training dataset with one of the automata with which a respective unmatched alert was correlated; and
program instructions to generate the new tree based on the selected features.

### Claim 17 (depends on 16)

The computer program product of claim 16, further comprising:
program instructions to prune the selected features based on a number of the unmatched alerts correlated with the automata via the paths including each feature and based on confidence values calculated for respective paths via which the unmatched alerts were correlated with the automata and which included a respective selected feature, wherein the confidence value is an estimation of a correlation accuracy; and
program instructions to generate the new tree based on unpruned features.

### Claim 18 (depends on 13)

The computer program product of claim 13, wherein the program instructions to analyze operation of the machine learning model further comprise:
responsive to applying the machine learning model to obtain a correlation result for a respective unmatched alert, program instructions to use a model-independent explanation technique to identify at least one element of the unmatched alert as an important element for the correlation result; and
program instructions to generate the new ruleset based on the at least one element of the unmatched alerts.

### Claim 19 (depends on 13)

The computer program product of claim 13, wherein the program instructions to analyze operation of the machine learning model further comprise:
when applying the machine learning model to correlate each unmatched alert, program instructions to calculate an initial confidence value for the initial correlation result, wherein the initial confidence value is an estimation of a correlation accuracy;
for different elements of the alert, program instructions to remove that element and reapply the machine learning model to the resulting alert to calculate a new confidence value for the initial correlation result;
program instructions to identify at least one element of the alert as an important element for the initial correlation result based on a difference between the initial confidence value and the new confidence value for the different elements; and
program instructions to generate the new ruleset based on the at least one element of the unmatched alerts.

### Claim 20

An alert-management system comprising a communications interface for receiving alerts from a data processing system, memory storing automated processes (“automata”) for handling the alerts and a master ruleset for matching the alerts with the automata, and control logic configured to:
train a machine learning model to correlate the alerts with the automata using a training dataset comprising matched alerts successfully handled by the automata;
apply the machine learning model to correlate unmatched alerts with automata, wherein unmatched alerts were not matched to the automata by the master ruleset;
analyze operation of the machine learning model in relation to correlation of the unmatched alerts to define a new ruleset for matching the unmatched alerts with the automata;
output the new ruleset for auditing of each rule in the new ruleset; and
responsive to approval of an audited rule, add the audited rule to the master ruleset.