# US 8,234,256 B2 · System and method for parsing, summarizing and reporting log data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,234,256 B2
- Document ID: US8234256B2
- Status: Granted
- Kind: B2
- Application number: 10/992,230
- Filed: 2004-11-18
- Published: 2005-05-26
- Granted: 2012-07-31
- Earliest claimed: 2003-11-26
- Assignees: LogLogic, Inc.
- Inventors: Jason Michael DeStefano; Thomas Hunt Schabo Grabowski
- Web page: https://patentgrep.com/patent/US8234256B2

## CPC

- G06F 40/205: Parsing
- G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)}
- G06F 11/3082: {the data filtering being achieved by aggregating or compressing the monitored data}
- H04L 41/069: using logs of notifications; Post-processing of notifications
- G06F 16/90: Details of database functions independent of the retrieved data types
- H04L 63/1425: {Traffic logging, e.g. anomaly detection}
- Y10S 707/99933: Query processing, i.e. searching
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

A system and method is disclosed which enables network administrators and the like to quickly analyze the data produced by log-producing devices such as network firewalls and routers. Unlike systems of the prior art, the system disclosed herein automatically parses and summarizes log data before inserting it into one or more databases. This greatly reduces the volume of data stored in the database and permits database queries to be run and reports generated while many types of attempted breaches of network security are still in progress. Database maintenance may also be accomplished automatically by the system to delete or archive old log data.

## Claims

### Claim 1

A method performed by a data processing system, comprising:
receiving raw log data from one or more log producing devices;
generating database statements from data fields extracted from the raw log data;
creating a first database table for storing the database statements, including designating a lifetime of the first database table such that when the lifetime of the first database table expires, a replacement first database table is created;
at an interval that corresponds to the lifetime, summarizing the database statements stored in the first database table into a data chunk that includes summary database statements, including:
identifying database statements stored in the first database table that share at least one common data field; and
combining the identified database statements into the summary database statements; and
storing, on a storage device, the summarized database statements as log files of the one or more log producing devices.

### Claim 2 (depends on 1)

The method of claim 1, comprising:
determining a data source of the raw log data; and
comparing the data source with a list of acceptable data sources before generating the one or more database statements.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the one or more database statements comprises:
searching the raw log data for a predetermined keyword to identify a message type; and
extracting the data field from the raw log data according to the message type.

### Claim 4 (depends on 1)

The method of claim 1, wherein the at least one common data field includes a data field of a source Internet protocol (IP) address, a destination IP address or a destination port number.

### Claim 5 (depends on 1)

The method of claim 1, wherein the summarized database statements as log files comprises:
storing the data chunk in a second database table that is associated with a first time period; and
periodically aggregating data chunks in the second database table to a third database table that is associated with a second time period that is longer than the first time period, including generating data that have coarser granularity than data in the data chunk.

### Claim 6 (depends on 5)

The method of claim 5, wherein:
the first database table includes a database table stored in memory, and
each of the second and third database tables includes at least one of an accept table or a deny table stored on disk.

### Claim 7

Computer instructions stored on a non-transitory medium, the computer instructions configured to cause a data processing system to perform operations comprising:
receiving raw log data from one or more log producing devices;
generating database statements from data fields extracted from the raw log data;
creating a first database table for storing the database statements, including designating a lifetime of the first database table such that when the lifetime of the first database table expires, a replacement first database table is created;
at an interval that corresponds to the lifetime, summarizing the database statements stored in the first database table into a data chunk that includes summary database statements, including:
identifying database statements stored in the first database table that share at least one common data field; and
combining the identified database statements into the summary database statements; and
storing, on a storage device, the summarized database statements as log files of the one or more log producing devices.

### Claim 8 (depends on 7)

The computer instructions of claim 7, the operations comprising:
determining a data source of the raw log data; and
comparing the data source with a list of acceptable data sources before generating the one or more database statements.

### Claim 9 (depends on 7)

The computer instructions of claim 7, wherein generating the one or more database statements comprises:
searching the raw log data for a predetermined keyword to identify a message type; and
extracting the data field from the raw log data according to the message type.

### Claim 10 (depends on 7)

The computer instructions of claim 7, wherein the at least one common data field includes a data field of a source Internet protocol (IP) address, a destination IP address or a destination port number.

### Claim 11 (depends on 7)

The computer instructions of claim 7, wherein the summarized database statements as log files comprises:
storing the data chunk in a second database table that is associated with a first time period; and
periodically aggregating data chunks in the second database table to a third database table that is associated with a second time period that is longer than the first time period, including generating data that have coarser granularity than data in the data chunk.

### Claim 12 (depends on 11)

The computer instructions of claim 11, wherein:
the first database table includes a database table stored in memory, and
each of the second and third database tables includes at least one of an accept table or a deny table stored on disk.

### Claim 13

A system, comprising:
one or more security servers connected to one or more log producing devices through a local area network, the one or more security servers configured to perform operations comprising:
receiving raw log data from the one or more log producing devices;
generating database statements from data fields extracted from the raw log data;
creating a first database table for storing the database statements, including designating a lifetime of the first database table such that when the lifetime of the first database table expires, a replacement first database table is created;
at an interval that corresponds to the lifetime, summarizing the database statements stored in the first database table into a data chunk that includes summary database statements, including:
identifying database statements stored in the first database table that share at least one common data field; and
combining the identified database statements into the summary database statements; and
storing, on a storage device, the summarized database statements as log files of the one or more log producing devices.

### Claim 14 (depends on 13)

The system of claim 13, the operations comprising:
determining a data source of the raw log data; and
comparing the data source with a list of acceptable data sources before generating the one or more database statements.

### Claim 15 (depends on 13)

The system of claim 13, wherein generating the one or more database statements comprises:
searching the raw log data for a predetermined keyword to identify a message type; and
extracting the data field from the raw log data according to the message type.

### Claim 16 (depends on 13)

The system of claim 13, wherein the at least one common data field includes a data field of a source Internet protocol (IP) address, a destination IP address or a destination port number.

### Claim 17 (depends on 13)

The system of claim 13, wherein the summarized database statements as log files comprises:
storing the data chunk in a second database table that is associated with a first time period; and
periodically aggregating data chunks in the second database table to a third database table that is associated with a second time period that is longer than the first time period, including generating data that have coarser granularity than data in the data chunk.

### Claim 18 (depends on 17)

The system of claim 17, wherein:
the first database table includes a database table stored in memory, and
each of the second and third database tables includes at least one of an accept table or a deny table stored on disk.