# US 2022/0277176 A1 · Log classification using machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277176 A1
- Document ID: US20220277176A1
- Status: Published application
- Kind: A1
- Application number: 17/187,137
- Filed: 2021-02-26
- Published: 2022-09-01
- Earliest claimed: 2021-02-26
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Aankur Bhatia; HuyAnh Dinh Ngo; Srinivas Babu Tummalapenta; Mahbod Tavallaee
- Web page: https://patentgrep.com/patent/US20220277176A1

## CPC

- G06F 18/241: relating to the classification model, e.g. parametric or non-parametric approaches
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods and a system of classifying unrecognized logs in an environment. The method includes inputting a log unrecognized during event collection into a machine learning model and predicting, by the machine learning model, a log source type of the log to allow for normalization of the log. The method also includes producing, by the machine learning model, a confidence score relating to the source type prediction, determining the confidence score exceeds a predetermined threshold, and submitting the log for normalization based on the log source type prediction. The method can also include predicting, by the machine learning model, an event name relating to the log, producing, by the machine learning model, a second confidence score relating to the event name prediction, determining the second confidence score exceeds another predetermined threshold, and submitting the log for normalization based on the identified log source type and the predicted event name.

## Claims

### Claim 1

A computer-implemented method of classifying unrecognized logs, the computer-implemented method comprising:
inputting a log unrecognized during event collection into a machine learning model, wherein the log relates to an event generated by a log source;
predicting, by the machine learning model, a log source type of the log to allow for normalization of the log;
producing, by the machine learning model, a confidence score relating to the source type prediction;
determining the confidence score exceeds a predetermined threshold; and
submitting the log for normalization based on the log source type prediction.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
predicting, by the machine learning model, an event name relating to the log;
producing, by the machine learning model, a second confidence score relating to the event name prediction;
determining the second confidence score exceeds another predetermined threshold; and
submitting the log for normalization based on the log source type prediction and the event name prediction.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the machine learning model is a one-dimensional convolutional neural network.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the convolutional neural network recognizes word patterns within the log.

### Claim 5 (depends on 3)

The computer-implemented method of claim 3, wherein the convolutional neural network is a six-layer network including three convolutional neural network (CNN) layers and two fully-connected layers, wherein the three CNN layers can each include both one-dimensional convolutions and sub-sampling pooling.

### Claim 6 (depends on 3)

The computer-implemented method of claim 3, wherein the convolutional neural network is trained using historical logs stored by a log collector, wherein the historical logs are previously known logs stored by the log collector.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the predetermined threshold is set by an administrator to allow for normalization of the log based on the confidence score achieving the predetermined threshold.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining the confidence scores do not exceed the predetermined threshold;
marking the log as unparsed; and
alerting an administrator that the log is unparsed.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining the log lacks available parsers based on the source type prediction; and
alerting an administrator that the log requires a new parser.

### Claim 10

A computer-implemented method of training a convolutional neural network to identify logs, the computer-implemented method comprising:
selecting a plurality of historical logs from a log storage, wherein the historical logs are previously known logs stored by a log collector;
preprocessing the historical logs into a labeled dataset for a convolutional neural network;
separating the labeled dataset into a training dataset and a validation dataset;
training the convolutional neural network with the training dataset to output log source types for unrecognized logs including confidence scores relating to the log source types; and
validating log source type predictions made by the convolutional neural network using the validation dataset.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein preprocessing the historical logs comprises:
parsing each of the historical logs to separate words located within the historical logs;
tokenizing the words in the historical logs by converting the words into numbers;
padding each of the numbers with additional zeroes such that each of the numbers is of a same byte length; and
vectorizing the numbers to be inputted into the convolutional neural network.

### Claim 12 (depends on 10)

The computer-implemented method of claim 10, wherein the convolutional neural network recognizes word patterns within the log.

### Claim 13 (depends on 10)

The computer-implemented method of claim 10, wherein the convolutional neural network is a six-layer network including three convolutional neural network (CNN) layers and two fully-connected layers, wherein the three CNN layers can each include both one-dimensional convolutions and sub-sampling pooling.

### Claim 14

A system of identifying logs, the system comprising:
a memory;
a processor;
a local data storage having stored thereon computer executable code;
a log storage configured to store log data including historical logs, wherein the historical logs are previously known logs;
an event collection module configured to identify and categorize logs;
a convolutional neural network configured to predict log source types relating to the logs unrecognized by the event collection module; and
a normalization component configured to apply parsers to normalize the identified logs.

### Claim 15 (depends on 14)

The system of claim 14, wherein the convolutional neural network recognizes word patterns within the log.

### Claim 16 (depends on 14)

The system of claim 14, wherein the convolutional neural network is a six-layer network including three convolutional neural network (CNN) layers and two fully-connected layers, wherein the three CNN layers can each include both one-dimensional convolutions and sub-sampling pooling.

### Claim 17 (depends on 14)

The system of claim 14, wherein the convolutional neural network outputs confidence scores relating to the log source types, wherein the confidence scores reflect a decimal number between zero and one interpreting a percentage of confidence for each of the log source type outputs.

### Claim 18 (depends on 14)

The system of claim 14, wherein the convolutional neural network is trained using the historical logs as training data.

### Claim 19 (depends on 14)

The system of claim 14, wherein the convolutional neural network outputs event names relating to the logs.

### Claim 20 (depends on 19)

The system of claim 19, wherein the convolutional neural network outputs additional confidence scores relating to the event names, wherein the additional confidence scores reflect a decimal number between zero and one interpreting a percentage of confidence for each of the log source type outputs.
