Automated log analysis and problem solving using intelligent operation and deep learning

A computer-implemented method of identifying one or more new application error types in a live computing session using a raw computer log file, the method comprising:

• loading into memory an error prediction engine, the error prediction engine including data clusters calculated based on historic computer log data using an artificial neural network, each cluster corresponding to a known error type;
• monitoring, using the error prediction engine, the raw computer log file for live input including application error and warning information;
• cross-referencing, using the error prediction engine, the live input with the data clusters stored in memory to predict whether a new application error type may occur;
• and
• generating, for each new error type that may occur, a corresponding warning message displayed during the live computing session,
• wherein the error prediction engine is built using a statistical pooling method to represent higher level statistical information associated with each unique computing session, the statistical pooling method generating a Z×W matrix of numerical entries representing W sessions, the W sessions each represented by a Z-dimensional vector reflecting a combination of Y actions in each session, each action including Z features, wherein Y, Z and W are each greater than one.

A computer-implemented method of determining, based on a raw computer log file, a root cause of at least one new application error, the method comprising:

• extracting from the raw computer log file a number of unstructured data sets, each unstructured data set associated with a unique session having a session identifier and including at least one error and at least two actions, each action including at least two features;
• processing each data set to generate a corresponding set of ordered frequent items;
• processing each set of ordered frequent items to generate a corresponding frequent pattern tree by:
• generating, for each unique new application error that is different from each known error type, a conditional pattern base of the unique error;
• and
• generating, based on each conditional pattern base, a conditional frequent pattern tree for each unique new application error;
• determining, based on each frequent pattern tree, a frequent set of actions associated with each unique new application error;
• and
• determining, based on each frequent set of actions associated with the error, the root cause of each new application error,
• wherein the at least one new application error is predicted using a statistical pooling method to represent higher level statistical information associated with each unique computing session, the statistical pooling method that generating a Z×W matrix of numerical entries representing W sessions, the W sessions each represented by a Z-dimensional vector reflecting a combination of Y actions in each session, each action including Z features, wherein Y, Z and W are each greater than one.

A computer-implemented method of training an error prediction model based on historic raw computer log data, the method comprising:

• processing the historic raw computer log data to generate unstructured data subsets, each unstructured data subset corresponding to a unique session having a first number of actions, each action having a second number of features;
• for each session ending in an error:
• extracting from each action a set of features;
• processing each set of features into a set of corresponding numerical components;
• generating, for each set of numerical components, a normalized vector;
• and
• grouping the normalized vectors into a Y×Z session matrix, wherein Y and Z are each greater than one;
• applying to each session matrix a statistical pooling method to represent higher level statistical information associated with each unique computing session, thereby generating a corresponding set of at least two Z-dimensional session vectors;
• and
• calculating, based on the set of session vectors, using an artificial neural network, a set of clusters, each cluster corresponding to a unique known error type, the set of clusters used to identify new error types that are different from each of the unique known error types corresponding to the calculated clusters.