Security system using automatic and scalable log pattern learning in security log analysis
AssigneeNEC Corporation
InventorsHui Zhang, Jianwu Xu, Bo Zong
Life of this patent
Estimating the term…
Abstract
A security system using automatic and scalable log pattern learning in security log analysis is provided. The security system includes one or more management services configured to generate security logs, and a security log analysis service operatively coupled to the one or more management services. The security log analysis service is configured to collect the security logs generated by the one or more management services, implement an incremental learning process to generate a set of log patterns from the collected security logs, parse the collected security logs using the set of log patterns, and analyze the parsed security logs for one or more security applications.
Claims20
A security system for providing computer system security, comprising:
- one or more management services implemented by at least one hardware component and configured to generate security logs, the one or more management services including a security event management (SEM) service and a security information management (SIM) service for implementing a security information and event management (SIEM) service;
- and
- a security log analysis service operatively coupled to the one or more management services, wherein the security log analysis service is configured to:
- collect the security logs generated by the one or more management services;
- implement an incremental learning process to generate a set of log patterns from the collected security logs;
- parse the collected security logs using the set of log patterns to obtain parsed security logs;
- and
- analyze the parsed security logs for one or more security applications;
- wherein, in implementing the incremental learning process to generate the set of log patterns, the security log analysis service is further configured to:
- define a first set as the training set, a second set as a set of log patterns that have been generated, and a third set as a set of logs of the training set that lack a matching pattern in the second set;
- sample the third set to generate a fourth set, wherein the fourth set has a size corresponding to a parameter controlling a maximum resource requirement for the incremental learning process;
- perform automatic log pattern recognition to generate a fifth set;
- and
- perform a log filtering process based on the fifth set.
A computer-implemented method for providing computer system security, comprising:
- collecting, by a processor operatively coupled to a memory, security logs generated by one or more management services of a security system, the one or more management services including a security event management (SEM) service and a security information management (SIM) service for implementing a security information and event management (SIEM) service;
- implementing, by the processor, an incremental learning process to generate a set of log patterns from the collected security logs;
- parsing, by the processor, the collected security logs using the set of log patterns to obtain parsed security logs;
- and
- analyzing, by the processor, the parsed security logs for one or more security applications;
- wherein implementing the incremental learning process to generate the set of log patterns further comprises:
- defining a first set as the training set, a second set as a set of log patterns that have been generated, and a third set as a set of logs of the training set that lack a matching pattern in the second set;
- sampling the third set to generate a fourth set, wherein the fourth set has a size corresponding to a parameter controlling a maximum resource requirement for the incremental learning process;
- performing automatic log pattern recognition to generate a fifth set;
- and
- performing a log filtering process based on the fifth set.
A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method for providing computer system security, comprising:
- collecting security logs generated by one or more management services of a security system, the one or more management services including a security event management (SEM) service and a security information management (SIM) service for implementing a security information and event management (SIEM) service;
- implementing an incremental learning process to generate a set of log patterns from the collected security logs;
- parsing the collected security logs using the set of log patterns to obtain parsed security logs;
- and
- analyzing the parsed security logs for one or more security applications;
- wherein implementing the incremental learning process to generate the set of log patterns further comprises:
- defining a first set as the training set, a second set as a set of log patterns that have been generated, and a third set as a set of logs of the training set that lack a matching pattern in the second set;
- sampling the third set to generate a fourth set, wherein the fourth set has a size corresponding to a parameter controlling a maximum resource requirement for the incremental learning process;
- performing automatic log pattern recognition to generate a fifth set;
- and performing a log filtering process based on the fifth set.
Dependencies are inferred from the claim text.