# US 11,443,168 B2 · Log analysis system employing long short-term memory recurrent neural net works

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,168 B2
- Document ID: US11443168B2
- Status: Granted
- Kind: B2
- Application number: 16/817,799
- Filed: 2020-03-13
- Published: 2021-09-16
- Granted: 2022-09-13
- Earliest claimed: 2020-03-13
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Vivek Srinivas; Bhavna Jindal
- Web page: https://patentgrep.com/patent/US11443168B2

## CPC

- G06N 3/08: Learning methods
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 11/0727: {in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097)}
- G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)}
- G06F 11/0766: {Error or fault reporting or storing}
- G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)}
- G06F 18/2137: based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
- G06F 18/24133: {Distances to prototypes}
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/09: Supervised learning
- H04L 63/1425: {Traffic logging, e.g. anomaly detection}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

System logs are processed to identify and report anomalies in execution of processes of a log-generating system such as a data storage system. Log messages of system logs are vectorized to generate log-message vectors; long short-term memory (LSTM) neural network processing is applied to the log-message vectors to generate an LSTM output sequence representing a production flow of the processes; and second-level neural network processing is applied to a combination of the LSTM output sequence and a training sequence to generate an analysis sequence containing a representation of anomalies in the production flow of the processes, where the training sequence is generated from a non-anomalous training flow. An anomaly report is generated and provided to a report consumer for taking further action with respect to the anomalies represented in the analysis sequence.

## Claims

### Claim 1

A method of processing system logs to identify and report anomalies in execution of one or more processes of a log-generating computerized system, comprising:
vectorizing log messages of the system logs to generate corresponding log-message vectors;
applying long short-term memory (LSTM) neural network processing to the log-message vectors to generate an LSTM output sequence representing a production flow of the processes;
applying second-level neural network processing to a combination of the LSTM output sequence and a training sequence to generate an analysis sequence containing a representation of anomalies in the production flow of the processes, the training sequence generated from a non-anomalous training flow of the processes; and
based on the analysis sequence, generating an anomaly report and providing it to a report consumer for taking further action with respect to the anomalies represented in the analysis sequence.

### Claim 2 (depends on 1)

The method of claim 1, wherein the vectorizing is a multiple-level vectorization including (1) a word-level tokenization and vectorization of individual words in the log messages, which produces a first representation in which each log message is represented as a respective sequence of word vectors, and (2) a message-level vectorization of the sequence of word vectors of each log message to generate a respective log-message vector for each distinct log message, the log-message vectors forming a set of representations of the log messages in a multi-dimensional log-message vector space.

### Claim 3 (depends on 2)

The method of claim 2, wherein the log-message vectors produced by the multiple-level vectorization are segregated into respective clusters in the multi-dimensional log-message vector space, with each cluster containing log-message vectors for log messages belonging to a corresponding process flow, and further including cluster processing to extract distinct sequences of log-message vectors for the respective process flows and apply the sequences to respective LSTMs processing for detecting anomalies in the respective processes.

### Claim 4 (depends on 1)

The method of claim 1, wherein the log-message vectors produced by the vectorizing are segregated into respective clusters in the multi-dimensional log-message vector space, with each cluster containing log-message vectors for log messages belonging to a corresponding process flow, and further including cluster processing to extract distinct sequences of log-message vectors for the respective process flows and apply the sequences to respective LSTMs processing for detecting anomalies in the respective processes.

### Claim 5

A log analyzer system having memory and processing circuitry, the memory storing computer program instructions executed by the processing circuitry to process system logs to identify and report anomalies in execution of one or more processes of a log-generating computerized system, the processing of the system logs including:
vectorizing log messages of the system logs to generate corresponding log-message vectors;
applying long short-term memory (LSTM) neural network processing to the log-message vectors to generate an LSTM output sequence representing a production flow of the processes;
applying second-level neural network processing to a combination of the LSTM output sequence and a training sequence to generate an analysis sequence containing a representation of anomalies in the production flow of the processes, the training sequence generated from a non-anomalous training flow of the processes; and
based on the analysis sequence, generating an anomaly report and providing it to a report consumer for taking further action with respect to the anomalies represented in the analysis sequence.

### Claim 6 (depends on 5)

The log analyzer system of claim 5, wherein the vectorizing is a multiple-level vectorization including (1) a word-level tokenization and vectorization of individual words in the log messages, which produces a first representation in which each log message is represented as a respective sequence of word vectors, and (2) a message-level vectorization of the sequence of word vectors of each log message to generate a respective log-message vector for each distinct log message, the log-message vectors forming a set of representations of the log messages in a multi-dimensional log-message vector space.

### Claim 7 (depends on 6)

The log analyzer system of claim 6, wherein the log-message vectors produced by the multiple-level vectorization are segregated into respective clusters in the multi-dimensional log-message vector space, with each cluster containing log-message vectors for log messages belonging to a corresponding process flow, and further including cluster processing to extract distinct sequences of log-message vectors for the respective process flows and apply the sequences to respective LSTMs processing for detecting anomalies in the respective processes.

### Claim 8 (depends on 5)

The log analyzer system of claim 5, wherein the log-message vectors produced by the vectorizing are segregated into respective clusters in the multi-dimensional log-message vector space, with each cluster containing log-message vectors for log messages belonging to a corresponding process flow, and further including cluster processing to extract distinct sequences of log-message vectors for the respective process flows and apply the sequences to respective LSTMs processing for detecting anomalies in the respective processes.