Pluggable join framework for stream processing
AssigneeYahoo Assets LLC
InventorsDavid Willcox, Maulik Shah, Allie K. Watfa, George Aleksandrovich
Life of this patent
Estimating the term…
Abstract
Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising receiving an event, such as a streaming event, from a client. The method determines that the event comprises a primary event and, if so, writes the primary event to a cache and returning the primary event to the client. The method later receives a second event from the client, the second event associated with the first event, annotates the second event based on the primary event, and returns the annotated second event to the client.
Claims20
A method comprising:
- receiving a primary event, the primary event including a join key;
- identifying a secondary event associated with the primary event using the join key;
- annotating the secondary event with fields from the primary event;
- marking the secondary event as late-joined;
- storing the primary event in a backing store;
- marking the primary event and the secondary event as out-of-order;
- and
- returning the primary event and the secondary event to a client.
A non-transitory computer-readable storage medium for tangibly storing computer program instructions capable of being executed by a computer processor, the computer program instructions defining steps of:
- receiving a primary event, the primary event including a join key;
- identifying a secondary event associated with the primary event using the join key;
- annotating the secondary event with fields from the primary event;
- marking the secondary event as late-joined;
- storing the primary event in a backing store;
- marking the primary event and the secondary event as out-of-order;
- and
- returning the primary event and the secondary event to a client.
A device comprising:
- a processor;
- and
- a storage medium for tangibly storing thereon logic for execution by the processor, the logic comprising instructions for:
- receiving a primary event, the primary event including a join key;
- identifying a secondary event associated with the primary event using the join key;
- annotating the secondary event with fields from the primary event;
- marking the secondary event as late-joined;
- storing the primary event in a backing store;
- marking the primary event and the secondary event as out-of-order;
- and
- returning the primary event and the secondary event to a client.
Dependencies are inferred from the claim text.