US 2014/0156683 A1 · INTEGRATING EVENT PROCESSING WITH MAP-REDUCE

Abstract

Large quantities of data can be processed and/or queried relatively quickly using a combination of continuous event processing and a Map-Reduce algorithmic tool. The continuous event processor can continuously produce real-time results by merging (a) CQL query results from events received since a currently executing Map-Reduce job was started with (b) a most recent query result produced by a most recently completed Map-Reduce job. When the currently executing Map-Reduce job completes, its query result can be stored and made accessible to the continuous event processor, and a new Map-Reduce job can be started relative to event data that has grown in size since the execution of the last Map-Reduce job. The Map-Reduce algorithmic tool provides a convenient mechanism for analyzing and processing large quantities of data.

Claims20

Dependent claims

A computer-implemented method comprising:

starting a first batch operation that executes a first query relative to event data that has been stored in a persistent data store as of a first time;
while the first batch operation is executing, continuously executing a second query relative to event data that is continuously being received from a data stream;
while the first batch operation is executing, continuously merging results of the second query with a result of a second batch operation that has completed;
continuously outputting results of the merging;
and
in response to a completion of the first batch operation, replacing the result of the second batch operation with a result of the first batch operation.

3 more elements

A computer-readable memory storing particular instructions to cause one or more processors to perform operators, the particular instructions comprising:

instructions to cause the one or more processors to start a first batch operation that executes a first query relative to event data that has been stored in a persistent data store as of a first time;
instructions to cause the one or more processors to continuously execute, while the first batch operation is executing, a second query relative to event data that is continuously being received from a data stream;
instructions to cause the one or more processors to continuously merge, while the first batch operation is executing, results of the second query with a result of a second batch operation that has completed;
instructions to cause the one or more processors to continuously output results of the merging;
and
instructions to cause the one or more processors to replace, in response to a completion of the first batch operation, the result of the second batch operation with a result of the first batch operation.

3 more elements

A system comprising:

one or more processors;
and
a computer-readable memory storing particular instructions to cause the one or more processors to perform operators, the particular instructions comprising:
instructions to cause the one or more processors to start a first batch operation that executes a first query relative to event data that has been stored in a persistent data store as of a first time;
instructions to cause the one or more processors to continuously execute, while the first batch operation is executing, a second query relative to event data that is continuously being received from a data stream;
instructions to cause the one or more processors to continuously merge, while the first batch operation is executing, results of the second query with a result of a second batch operation that has completed;
instructions to cause the one or more processors to continuously output results of the merging;
and
instructions to cause the one or more processors to replace, in response to a completion of the first batch operation, the result of the second batch operation with a result of the first batch operation.

6 more elements

Dependencies are inferred from the claim text.

Integrating event processing with map-reduce

Life of this patent

Integrating event processing with map-reduce

Life of this patent

Abstract

Claims20