Back to results
US 12,585,567 B2Granted
System and method for generating one or more aggregation rules configured to be utilized by a log collector
AssigneeCyberProof Israel Ltd.
InventorsEliran Dror, Odelia Cerf
Life of this patent
Estimating the term…
Abstract
The presently disclosed subject matter aims to a system and method for generating one or more aggregation rules configured to be utilized by a log collector. The system and method involve an aggregation rules generator directed to allow a log collector to handle the aggregation of gathered log information, even for event logs with previously unobserved behavior patterns.
Claims20
A system for generating one or more aggregation rules configured to be utilized by a log collector, the system comprising:
- a network interface configured to receive, from one or more external log sources, a plurality of records, each record including a plurality of values, wherein each value of the values is associated with a respective feature of a plurality of features;
- a data repository configured to store the plurality of records, features, associated values, entropy scores, and aggregation rules;
- a process circuitry operatively coupled to the network interface and the data repository, the processing circuitry including an aggregation rule generator module, the processing circuitry configured to:
- obtain a plurality of records from the data repository;
- calculate, for at least two of the features, an entropy score based on the values of the plurality of records associated with the respective feature, wherein the entropy score is indicative of an entropy level of the values of the plurality of records associated with the respective feature;
- identify:
- (a) one or more low-entropy features of the features, wherein the entropy score of the one or more low-entropy features is below a first threshold, and (b) one or more repeating sequences of values associated with the low-entropy features, wherein the repeating sequences exist within a plurality of given records of the plurality of records;
- calculate for each repeating sequence of the one or more repeating sequences a repeating sequence score, wherein the repeating sequence score is based on a number of the given records;
- and
- generate one or more aggregation rules usable by the log collector to aggregate repeating sequences of the one or more repeating sequences having a repeating sequence score that is above a second threshold,
- wherein each of the one or more aggregation rules is deployed to the log collector for execution in real-time and is automatically updated in response to periodic reassessment of live operational log data or based on outcomes from a machine learning model trained to predict aggregation time windows.
8 more elements
A method performed by a system comprising a network interface, data repository, and processing circuitry including an aggregation rule generator module for generating one or more aggregation rules configured to be utilized by a log collector, the method comprising:
- obtaining a plurality of records via the network interface and stored in the data repository, each including a plurality of values, wherein each value of the values is associated with a respective feature of a plurality of features;
- calculating, for at least two of the features, an entropy score based on the values of the plurality of records associated with the respective feature, wherein the entropy score is indicative of an entropy level of the values of the plurality of records associated with the respective feature;
- identifying:
- (a) one or more low-entropy features of the features, wherein the entropy score of the one or more low-entropy features is below a first threshold, and (b) one or more repeating sequences of values associated with the low entropy features, wherein the repeating sequences exist within a plurality of given records of the plurality of records;
- calculating for each repeating sequence of the one or more repeating sequences a repeating sequence score, wherein the repeating sequence score is based on a number of the given records;
- and
- generating one or more aggregation rules usable by the log collector to aggregate repeating sequences of the one or more repeating sequences having a repeating sequence score that is above a second threshold,
- wherein each of the one or more generated aggregation rules is automatically deployed in real-time to the log collector and is periodically updated based on live operational log data or results from a machine learning model trained to predict aggregation windows.
5 more elements
A non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at least one processor of a system comprising a network interface, data repository, and processing circuitry including an aggregation rule generator module to perform a method comprising:
- receiving, via the network interface, a plurality of records from one or more external log sources and storing the records in the data repository, wherein each value of the values is associated with a respective feature of a plurality of features;
- calculating, for at least two of the features, an entropy score based on the values of the plurality of records associated with the respective feature, wherein the entropy score is indicative of an entropy level of the values of the plurality of records associated with the respective feature;
- identifying:
- (a) one or more low-entropy features of the features, wherein the entropy score of the one or more low-entropy features is below a first threshold, and (b) one or more repeating sequences of values associated with the low-entropy features, wherein the repeating sequences exist within a plurality of given records of the plurality of records;
- calculating for each repeating sequence of the one or more repeating sequences a repeating sequence score, wherein the repeating sequence score is based on a number of the given records;
- and
- generating one or more aggregation rules usable by the log collector to aggregate repeating sequences of the one or more repeating sequences having a repeating sequence score that is above a second threshold.
4 more elements
Dependencies are inferred from the claim text.