Multibyte heterogeneous log preprocessing
AssigneeNEC Corporation
InventorsJianwu Xu, Biplob Debnath, Hui Zhang, Guofei Jiang
Life of this patent
Estimating the term…
Abstract
Methods and systems for log management include pre-processing heterogeneous logs and performing a log management action on the pre-processed plurality of heterogeneous logs. Pre-processing the logs includes performing a fixed tokenization of the heterogeneous logs based on a predefined set of symbols, performing a flexible tokenization of the heterogeneous logs based on a user-defined set of rules, converting timestamps in the heterogeneous logs to a single target timestamp format, and performing structural log tokenization of the heterogeneous logs based on user-defined structural information.
Claims16
A method for log management, comprising:
- pre-processing a plurality of heterogeneous logs, said pre-processing comprising:
- performing a fixed tokenization of a plurality of heterogeneous logs based on a predefined set of symbols;
- performing a flexible tokenization of the plurality of heterogeneous logs based on a user-defined set of rules;
- converting timestamps in the plurality of heterogeneous logs to a single target timestamp format;
- and
- performing structural log tokenization of the plurality of heterogeneous logs based on user-defined structural information;
- and
- performing a log management action on the pre-processed plurality of heterogeneous logs.
A log management system, comprising:
- a fixed tokenization module configured to perform a fixed tokenization of a plurality of heterogeneous logs based on a predefined set of symbols;
- a flexible tokenization module configured to perform a flexible tokenization of the plurality of heterogeneous logs based on a user-defined set of rules;
- a timestamp conversion module configured to convert timestamps in the plurality of heterogeneous logs to a single target timestamp format;
- a structural tokenization module configured to perform structural log tokenization of the plurality of heterogeneous logs based on user-defined structural information;
- and
- a log analysis module configured to perform a log management action on the pre-processed plurality of heterogeneous logs.
Dependencies are inferred from the claim text.