Customized reporting and mining of event data
AssigneeLogLogic, Inc.
InventorsSherif Botros, Jian L. Zhen, Minjun Liu, Boris Galitsky
Life of this patent
Estimating the term…
Abstract
Event data (e.g., log messages) are represented as sets of attribute/value pairs. An index maps each attribute/value pair or attribute/value tuple to a pointer that points to event data which contains the attribute/value pair or attribute/value tuple. An attribute co-occurrence map or matrix can be generated that includes attribute names that co-occur together. Queries and custom reports can be generated by projecting event data into one or more attributes or attribute/value pairs, and then determining statistics on other attributes using a combination of the inverted index, the attribute co-occurrence map or matrix, operations on sets and/or math and statistical functions.
Claims19
A computer-implemented method, comprising:
- receiving event data in a log file;
- transforming the event data into attribute/value pairs;
- generating an index mapping at least one of the attribute/value pairs to a reference pointer that references an instance of event data in the log file, the instance of event data in the log file including a textual message from which the attribute/value pair was transformed;
- generating an attribute co-occurrence data structure, the attribute co-occurrence data structure identifying a relationship between a first attribute in the attribute/value pairs and a second attribute in the attribute/value pairs, where existence of the relationship between the first attribute and the second attribute indicates that reports for finding a textual message including both the first attribute and the second attribute are allowable;
- and
- in response to a query including the first attribute, generating a report based on the attribute co-occurrence data structure and the attribute/value pairs in the index, including:
- identifying the second attribute based on the relationship identified in the attribute co-occurrence data structure;
- identifying an attribute/value pair that includes the second attribute;
- and
- generating the report, including providing access to the event data in the log file using a reference pointer that is mapped to the identified attribute/value pair in the index,
- where the method is performed by one or more processors.
A system, comprising:
- a storage device operable for storing event data in a log file;
- and
- one or more data processing devices configured to perform operations comprising:
- transforming the event data into attribute/value pairs;
- generating an index mapping at least one of the attribute/value pairs to a reference pointer that references an instance of event data in the log file, the instance of event data in the log file including a textual message from which the attribute/value pair was transformed;
- generating an attribute co-occurrence data structure, the attribute co-occurrence data structure identifying a relationship between a first attribute in the attribute/value pairs and a second attribute in the attribute/value pairs, where existence of the relationship between the first attribute and the second attribute indicates that reports for finding a textual message including both the first attribute and the second attribute are allowable;
- and
- in response to a query including the first attribute, generating a report based on the attribute co-occurrence data structure and the attribute/value pairs in the index, including:
- identifying the second attribute based on the relationship identified in the attribute co-occurrence data structure;
- identifying an attribute/value pair that includes the second attribute;
- and
- generating the report, including providing access to the event data in the log file using a reference pointer that is mapped to the identified attribute/value pair in the index.
Dependencies are inferred from the claim text.