Troubleshooting based on log similarity

An apparatus for trouble shooting based on log similarity, comprising:

• a processor;
• memory in communication with the processor;
• a log module, via the processor and memory, extracting log patterns from a plurality of log files in response to having collected the plurality of log files from at least one system with troubles, each log pattern describing a regular expression to which a log message in a log file among the plurality of log files conforms;
• a building module, via the processor and memory, building a pattern repository using the log patterns extracted;
• a mapping module, via the processor and memory, mapping each of the plurality of log files to an n-dimensional vector based on the pattern repository;
• and
• a clustering module, via the processor and memory, clustering a plurality of n-dimensional vectors to which each of the plurality of log files is mapped into at least one group, wherein each of the at least one group indicates one trouble type of the at least one system;
• wherein the mapping module is configured to, with respect to a log file j among the plurality of log files, match each line of log message k in the log file j to a corresponding log pattern pk in the pattern repository;
• transform the log file j into a sequence fj of the corresponding log pattern pk;
• and
• map the sequence fj to an n-dimensional vector;
• wherein dimension n of the n-dimensional vector is proportional to the amount of log patterns in the pattern repository;
• wherein the mapping module is configured to, with respect to a log pattern pi in the pattern repository, calculate an eigenvalue tfidfi,j wherein tfidfi,j=tfi,j×idfi;
• the eigenvalue tfidfi,j associated with the sequence fj and the log pattern pi, and the eigenvalue tfidfi,j associated with an occurrence frequency of the log pattern pi in multiple sequences corresponding to the plurality of log files;
• and
• further configured to treat the eigenvalue tfidfi,j as the ith component in the n-dimensional vector to build the n-dimensional vector.