# US 9,519,535 B1 · Two stage log normalization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,535 B1 - Document ID: US9519535B1 - Status: Granted - Kind: B1 - Application number: 14/802,622 - Filed: 2015-07-17 - Granted: 2016-12-13 - Earliest claimed: 2015-07-17 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Phillip A. J. Cooper; Jevon J. C. Hill; Fiona L. Lam; Kalvinder P. Singh - Web page: https://patentgrep.com/patent/US9519535B1 ## CPC - G06F 11/0781: {Error filtering or prioritizing based on a policy defined by the user or on a policy defined by a hardware/software module, e.g. according to a severity level} - G06F 11/0706: {the processing taking place on a specific hardware platform or in a specific software environment} - G06F 11/079: {Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36)} - G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)} - H04L 41/16: using machine learning or artificial intelligence - H04L 43/04: Processing captured monitoring data, e.g. for logfile generation - H04L 43/50: Testing arrangements - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract An approach for two stage log normalization is provided. The approach retrieves a message format and a plurality of parameters from one or more log files. The approach determines a classification for one or more first sequence files, wherein the one or more first sequence files includes the message format from the one or more log files. The approach determines a classification of error for the one or more first sequence files. The approach determines whether there is a high confidence in the classification of error for the one or more first sequence files. The approach determines whether there is an improvement in confidence in the classification of error from one or more second sequence files, wherein the one or more second sequence files includes the message format and the plurality of parameters from the one or more log files. ## Claims ### Claim 1 A method for two stage log normalization, the method comprising: retrieving, by one or more computer processors, a message format and a plurality of parameters from one or more log files; determining, by one or more computer processors, a classification for one or more first sequence files, wherein the one or more first sequence files includes the message format from the one or more log files; determining, by one or more computer processors, a classification of error for the one or more first sequence files; determining, by one or more computer processors, whether there is a high confidence in the classification of error for the one or more first sequence files; and responsive to a determination that there is not a high confidence in the classification of error for the one or more first sequence files, determining, by one or more computer processors, whether there is an improvement in confidence in the classification of error from one or more second sequence files, wherein the one or more second sequence files includes the message format and the plurality of parameters from the one or more log files, wherein determining whether there is an improvement in confidence in the classification of error from the one or more second sequence files includes: creating the one or more second sequence files, wherein creating includes retrieving a corresponding unique parameter identifier (ID) for each of the one or more log files and sequencing a corresponding unique message ID for each of the one or more log files with the respective unique parameter ID; re-classifying the one or more second sequence files, wherein re-classifying includes determining a classification of error for the one or more second sequence files; and determining a level of similarity between the classification of error for the one or more second sequence files and a plurality of existing trained data. ### Claim 2 (depends on 1) The method of claim 1, wherein determining a classification for one or more first sequence files, further comprises: creating, by one or more computer processors, the one or more first sequence files, wherein creating includes retrieving the corresponding unique message identifier (ID) for each of the one or more log files; and utilizing, by one or more computer processors, conventional machine learning processes to train data for the classification of the one or more first sequence files. ### Claim 3 (depends on 1) The method of claim 1, wherein determining a classification of error for the one or more first sequence files, further comprises: determining, by one or more computer processors, one or more similarities between the one or more first sequence files and a plurality of existing trained data. ### Claim 4 (depends on 1) The method of claim 1, wherein determining whether there is a high confidence in the classification of error for the one or more first sequence files, further comprises: determining, by one or more computer processors, a level of similarity between the classification of error for the one or more first sequence files and a plurality of existing trained data. ### Claim 5 (depends on 4) The method of claim 4, further comprising: responsive to a determination that the level of similarity between the classification of error for the one or more first sequence files and the plurality of existing trained data is high, determining, by one or more computer processors, that there is a high confidence in the classification of error; and responsive to a determination that the level of similarity between the classification of error for the one or more first sequence files and the plurality of existing trained data is low, determining, by one or more computer processors, that there is not a high confidence in the classification of error. ### Claim 6 (depends on 1) The method of claim 1, further comprising: responsive to a determination that the level of similarity between the classification of error for the one or more second sequence files and the plurality of existing trained data has improved over a level of similarity between a classification of error for the one or more first sequence files and the plurality of existing trained data, determining, by one or more computer processors, there is an improvement in confidence in the classification of error for the one or more second sequence files; and responsive to a determination that the level of similarity between the classification of error for the one or more second sequence files and the plurality of existing trained data has not improved over a level of similarity between a classification of error for the one or more first sequence files and the plurality of existing trained data, determining, by one or more computer processors, there is not an improvement in confidence in the classification of error for the one or more second sequence files. ### Claim 7 A computer program product for two stage log normalization, the computer program product comprising: one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising: program instructions to retrieve a message format and a plurality of parameters from one or more log files; program instructions to determine a classification for one or more first sequence files, wherein the one or more first sequence files includes the message format from the one or more log files; program instructions to determine a classification of error for the one or more first sequence files; program instructions to determine whether there is a high confidence in the classification of error for the one or more first sequence files; and responsive to a determination that there is not a high confidence in the classification of error for the one or more first sequence files, program instructions to determine whether there is an improvement in confidence in the classification of error from one or more second sequence files, wherein the one or more second sequence files includes the message format and the plurality of parameters from the one or more log files, wherein determining whether there is an improvement in confidence in the classification of error from the one or more second sequence files includes: creating the one or more second sequence files, wherein creating includes retrieving a corresponding unique parameter identifier (ID) for each of the one or more log files and sequencing a corresponding unique message ID for each of the one or more log files with the respective unique parameter ID; re-classifying the one or more second sequence files, wherein re-classifying includes determining a classification of error for the one or more second sequence files; and determining a level of similarity between the classification of error for the one or more second sequence files and a plurality of existing trained data. ### Claim 8 (depends on 7) The computer program product of claim 7, wherein program instructions to determine a classification for one or more first sequence files, further comprises: program instructions to create the one or more first sequence files, wherein creating includes retrieving the corresponding unique message identifier (ID) for each of the one or more log files; and program instructions to utilize conventional machine learning processes to train data for the classification of the one or more first sequence files. ### Claim 9 (depends on 7) The computer program product of claim 7, wherein program instructions to determine a classification of error for the one or more first sequence files, further comprises: program instructions to determine one or more similarities between the one or more first sequence files and a plurality of existing trained data. ### Claim 10 (depends on 7) The computer program product of claim 7, wherein program instructions to determine whether there is a high confidence in the classification of error for the one or more first sequence files, further comprises: program instructions to determine a level of similarity between the classification of error for the one or more first sequence files and a plurality of existing trained data. ### Claim 11 (depends on 10) The computer program product of claim 10, further comprising: responsive to a determination that the level of similarity between the classification of error for the one or more first sequence files and the plurality of existing trained data is high, program instructions to determine that there is a high confidence in the classification of error; and responsive to a determination that the level of similarity between the classification of error for the one or more first sequence files and the plurality of existing trained data is low, program instructions to determine that there is not a high confidence in the classification of error. ### Claim 12 (depends on 7) The computer program product of claim 7, further comprising: responsive to a determination that the level of similarity between the classification of error for the one or more second sequence files and the plurality of existing trained data has improved over a level of similarity between a classification of error for the one or more first sequence files and the plurality of existing trained data, program instructions to determine there is an improvement in confidence in the classification of error for the one or more second sequence files; and responsive to a determination that the level of similarity between the classification of error for the one or more second sequence files and the plurality of existing trained data has not improved over a level of similarity between a classification of error for the one or more first sequence files and the plurality of existing trained data, program instructions to determine there is not an improvement in confidence in the classification of error for the one or more second sequence files. ### Claim 13 A computer system for two stage log normalization, the computer system comprising: one or more computer processors; one or more computer readable storage media; program instructions stored on at least one of the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising: program instructions to retrieve a message format and a plurality of parameters from one or more log files; program instructions to determine a classification for one or more first sequence files, wherein the one or more first sequence files includes the message format from the one or more log files; program instructions to determine a classification of error for the one or more first sequence files; program instructions to determine whether there is a high confidence in the classification of error for the one or more first sequence files; and responsive to a determination that there is not a high confidence in the classification of error for the one or more first sequence files, program instructions to determine whether there is an improvement in confidence in the classification of error from one or more second sequence files, wherein the one or more second sequence files includes the message format and the plurality of parameters from the one or more log files, wherein determining whether there is an improvement in confidence in the classification of error from the one or more second sequence files includes: creating the one or more second sequence files, wherein creating includes retrieving a corresponding unique parameter identifier (ID) for each of the one or more log files and sequencing a corresponding unique message ID for each of the one or more log files with the respective unique parameter ID; re-classifying the one or more second sequence files, wherein re-classifying includes determining a classification of error for the one or more second sequence files; and determining a level of similarity between the classification of error for the one or more second sequence files and a plurality of existing trained data. ### Claim 14 (depends on 13) The computer system of claim 13, wherein program instructions to determine a classification for one or more first sequence files, further comprises: program instructions to create the one or more first sequence files, wherein creating includes retrieving the corresponding unique message identifier (ID) for each of the one or more log files; and program instructions to utilize conventional machine learning processes to train data for the classification of the one or more first sequence files. ### Claim 15 (depends on 13) The computer system of claim 13, wherein program instructions to determine whether there is a high confidence in the classification of error for the one or more first sequence files, further comprises: program instructions to determine a level of similarity between the classification of error for the one or more first sequence files and a plurality of existing trained data. ### Claim 16 (depends on 15) The computer system of claim 15, further comprising: responsive to a determination that the level of similarity between the classification of error for the one or more first sequence files and the plurality of existing trained data is high, program instructions to determine that there is a high confidence in the classification of error; and responsive to a determination that the level of similarity between the classification of error for the one or more first sequence files and the plurality of existing trained data is low, program instructions to determine that there is not a high confidence in the classification of error. ### Claim 17 (depends on 13) The computer system of claim 13, further comprising: responsive to a determination that the level of similarity between the classification of error for the one or more second sequence files and the plurality of existing trained data has improved over a level of similarity between a classification of error for the one or more first sequence files and the plurality of existing trained data, program instructions to determine there is an improvement in confidence in the classification of error for the one or more second sequence files; and responsive to a determination that the level of similarity between the classification of error for the one or more second sequence files and the plurality of existing trained data has not improved over a level of similarity between a classification of error for the one or more first sequence files and the plurality of existing trained data, program instructions to determine there is not an improvement in confidence in the classification of error for the one or more second sequence files.