Nlp-based context-aware log mining for troubleshooting

A method of context-aware data mining of a text document, comprising the steps of:

• parsing and preprocessing the text document and producing a tokenized word list;
• defining a word dictionary from the tokenized word list, wherein the word dictionary includes at least some of the tokens in the tokenized word list;
• and
• training the word embedding model, wherein the word embedding model is a neural network model that represents each word or line in the word dictionary by a vector,
• wherein parsing and preprocessing the text document comprises the steps of:
• removing all punctuation and a preamble from each line in the text document;
• parsing numeric data;
• tokenizing the text document in words to form a tokenized word list, wherein a token is one of a single word, an N-gram of N consecutive words, or an entire line of the document;
• and
• returning the tokenized word list.

A program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for context-aware data mining of a text document, comprising the steps of:

• receiving a list of words parsed and preprocessed from an input query;
• computing a related distributed embedding representation for each word in the list of words using a word embedding model of the text document being queried;
• aggregating the related distributed embedding representations of all words in the list of words to represent the input query with a single embedding, by using one of an average of all the related distributed embedding representations or a maximum of all the related distributed embedding representations;
• retrieving a ranked list of document segments of N lines that are similar to the aggregated word embedding representation of the query, wherein N is a positive integer provided by the user;
• and
• returning the list of retrieved segments to a user.