US 10,255,141 B2Granted

Event batching, output sequencing, and log based state storage in continuous query processing

AssigneeOracle International Corporation

InventorsHoyong Park, Sandeep Bishnoi, Unmesh Anil Deshmukh, Prabhu Thukkaram

Life of this patent

Estimating the term…

Loading patent…

Abstract

An event processing system is disclosed that processes events of an event stream and performs the recovery of events during system failure. The recovery of events is performed by generating output sequence numbers for each deterministic output event generated by the event processing system. In an embodiment, the event processing system determines a current output sequence number of a most recently transmitted output event in a output stream of events, a last output sequence number of an output event corresponding to a most recently processed batch of input events and determines a set of one or more output events of the output stream to be transmitted based on the current sequence number and the last output sequence number. The event processing system then transmits the output events related to the application.

Claims18

Dependent claims

A system comprising:

a memory storing a plurality of instructions;
and
one or more processors configured to access the memory, wherein the one or more processors are further configured to execute the plurality of instructions to at least:
receive a continuous input stream of events related to an application;
generate a checkpoint marker event;
determine an interval for inserting the checkpoint marker event, the interval based at least in part on a type of the application, a latency requirement of the application, and a frequency at which events of the input stream of events are received;
insert the checkpoint marker event into the continuous input stream of events at each interval;
generate a set of one or more event batches based at least in part on the checkpoint marker event;
determine a size for a particular event batch of the one or more event batches based at least in part on the interval for inserting the checkpoint marker event;
process the continuous input stream of events to generate an output stream of events related to the application;
determine an output sequence number for an output event in the output stream of events;
transmit the output event comprising the output stream of events;
store the output sequence number of the output event;
while the continuous input stream of events is being processed, receive an indication of failure of the system;
and
in response to the indication of failure:
determine a current output sequence number of a most recently transmitted output event in the output stream of events;
determine a last output sequence number of an output event corresponding to a most recently processed batch of the one or more event batches;
determine a set of one or more output events of the output stream to be transmitted based at least in part on the current sequence number and the last output sequence number;
and
transmit the set of one or more output events related to the application.

18 more elements

A method comprising:

receiving a continuous input stream of events related to an application;
generating a checkpoint marker event;
determining an interval for inserting the checkpoint marker event, the interval based at least in part on a type of the application, a latency requirement of the application, and a frequency at which events of the input stream of events are received;
inserting the checkpoint marker event into the continuous input stream of events at each interval;
generating a set of one or more event batches based at least in part on the checkpoint marker event;
determining a size for a particular event batch of the one or more event batches based at least in part on the interval for inserting the checkpoint marker event;
processing the continuous input stream of events to generate an output stream of events related to the application;
determining an output sequence number for an output event in the output stream of events;
transmitting the output event of the output stream of events;
storing the output sequence number of the output event;
while the continuous input stream of events is being processed, receiving an indication of failure of the system;
and
in response to the indication of failure:
determining a current output sequence number of a most recently transmitted output event in the output stream of events;
determining a last output sequence number of an output event corresponding to a most recently processed batch of the one or more event batches;
determining a set of one or more output events of the output stream to be transmitted based at least in part on the current sequence number and the last output sequence number;
and
transmitting the set of one or more output events related to the application.

15 more elements

A non-transitory computer-readable storage memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising:

instructions that cause the one or more processors to receive a continuous input stream of events related to an application;
instructions that cause the one or more processors to generate a checkpoint marker event;
instructions that cause the one or more processors to determine an interval for inserting the checkpoint marker event, the interval based at least in part on a type of the application and a latency requirement of the application;
instructions that cause the one or more processors to insert the checkpoint marker event into the continuous input stream of events at each interval;
instructions that cause the one or more processors to generate a set of one or more event batches based at least in part on the checkpoint marker event;
instructions that cause the one or more processors to determine a size for a particular event batch of the one or more event batches based at least in part on the interval for inserting the checkpoint marker event;
instructions that cause the one or more processors to process the continuous input stream of events to generate an output stream of events related to the application;
instructions that cause the one or more processors to determine an output sequence number for an output event in the output stream of events;
instructions that cause the one or more processors to transmit the output event of the output stream of events;
instructions that cause the one or more processors to store the output sequence number of the output event;
instructions that cause the one or more processors to receive an indication of failure of the system while the continuous input stream of events is being processed;
instructions that cause the one or more processors to determine a current output sequence number of a most recently transmitted output event in the output stream of events;
instructions that cause the one or more processors to determine a last output sequence number of an output event corresponding to a most recently processed batch of the one or more event batches;
instructions that cause the one or more processors to determine a set of one or more output events of the output stream to be transmitted based at least in part on the current sequence number and the last output sequence number;
and
instructions that cause the one or more processors to transmit the set of one or more output events related to the application.

13 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/161,043Filed 20 May 2016
Priority: IN 5688/CHE/201522 Oct 2015
Published: US 2017/0116089 A127 Apr 2017
Granted: 9 Apr 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G06F 11/1471
Error detection or correction of the data by redundancy in operations: involving logging of persistent data for recovery
G06F 9/542
Interprogram communication: event management; Broadcasting; Multicasting; Notifications
G06F 11/1438
At system level: restarting or rejuvenating
G06F 16/128
File system administration, e.g. details of archiving or snapshots: details of file system snapshots on the file-level, e.g. snapshot creation,…