US 2016/0103881 A1Published application

Partitioning log records based on term frequency and type for selective skipping during full-text searching

AssigneeCA, Inc.

InventorSreenivas Gukal

Life of this patent

Estimating the term…

Loading patent…

Abstract

A log record from a host machine node includes terms. Frequency of occurrence of the terms across a stream of log records is determined. Based on the frequency satisfying a threshold, a Bloom filter vector is selected from among a plurality of Bloom filter vectors based on the frequency, the Bloom filter vector is updated based on the terms, and an identifier for the log record is stored with an association to the Bloom filter vector. In contrast, based on the frequency of occurrence not satisfying the defined frequency range, a type identifier is identified based on the terms, a Bloom filter vector is selected from among the plurality of Bloom filter vectors based on the type identifier, the Bloom filter vector is updated based on the terms, and an identifier for the log record is stored with an association to the Bloom filter vector.

Claims20

Dependent claims

A method by a log record processing computer comprising:

receiving a log record as part of a stream of log records from a host machine node, the log record comprising terms;
determining frequency of occurrence of the terms across the log records of the stream;
based on the frequency of occurrence satisfying a defined frequency threshold:
selecting a frequency partitioned Bloom filter vector from among a plurality of Bloom filter vectors in a repository based on the frequency;
and
updating the frequency partitioned Bloom filter vector in the repository based on the terms;
storing an identifier for the log record associated with the frequency partitioned Bloom filter vector;
and
based on the frequency of occurrence not satisfying the defined frequency range:
identifying a type identifier based on the terms;
selecting a type identifier partitioned Bloom filter vector from among the plurality of Bloom filter vectors in the repository based on the type identifier;
updating the type identifier partitioned Bloom filter vector in the repository based on the terms;
and
storing an identifier for the log record associated with the type identifier partitioned Bloom filter vector.

11 more elements

A computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:
computer readable program code to receive a log record as part of a stream of log records from a host machine node, the log record comprising terms;
computer readable program code to determine frequency of occurrence of the terms across the log records of the stream;
computer readable program code to, based on the frequency of occurrence satisfying a defined frequency threshold, perform:
selecting a frequency partitioned Bloom filter vector from among a plurality of Bloom filter vectors in a repository based on the frequency;
and
updating the frequency partitioned Bloom filter vector in the repository based on the terms and associating;
storing an identifier for the log record associated with the frequency partitioned Bloom filter vector;
and
computer readable program code to, based on the frequency of occurrence not satisfying the defined, frequency range, perform:
identifying a type identifier based on the terms;
selecting a type identifier partitioned Bloom filter vector from among the plurality of Bloom filter vectors in the repository based on the type identifier;
updating the type identifier partitioned Bloom filter vector in the repository based on the terms;
and
storing an identifier for the log record associated with the type identifier partitioned Bloom filter vector.

12 more elements

Dependencies are inferred from the claim text.