Failure management for data streaming processing system
AssigneeAT&T Intellectual Property I, L.P.
InventorsTHEODORE JOHNSON, Vladislav Shkapenyuk
Life of this patent
Estimating the term…
Abstract
A device, method and computer-readable medium for providing failure management of a data streaming processing system are disclosed. A method may detect a first replica of a shard has failed during processing of the stream of data records between a first operator and a second operator, notify the first operator and the second operator that the first replica of the shard has failed, perform a rewind procedure to identify data records that have been processed in the first replica and resume processing of the stream of data records between the first operator and the second operator via a second replica of the shard without processing the data records that have already been processed from the first replica.
Claims20
A device comprising:
- a processor of a data stream processing system implementing a coordinator for processing a stream of data records;
- and
- a computer-readable medium storing instructions which, when executed by the processor, cause the processor to perform operations, the operations comprising:
- detecting a first replica of a shard has failed during processing of the stream of data records between a first operator and a second operator;
- notifying the first operator and the second operator that the first replica of the shard has failed;
- performing a rewind procedure to identify data records that have been processed in the first replica;
- and
- resuming processing of the stream of data records between the first operator and the second operator via a second replica of the shard without processing the data records that have already been processed from the first replica.
A method comprising:
- detecting, by a processor, a first replica of a shard has failed during processing of the stream of data records between a first operator and a second operator;
- notifying, by the processor, the first operator and the second operator that the first replica of the shard has failed;
- performing, by the processor, a rewind procedure to identify data records that have been processed in the first replica;
- and
- resuming, by the processor, processing of the stream of data records between the first operator and the second operator via a second replica of the shard without processing the data records that have already been processed from the first replica.
A non-transitory computer-readable medium storing instructions which, when executed by a processor of a data stream processing system implementing a coordinator for processing a stream of data records, cause the processor to perform operations, the operations comprising:
- detecting a first replica of a shard has failed during processing of the stream of data records between a first operator and a second operator;
- notifying the first operator and the second operator that the first replica of the shard has failed;
- performing a rewind procedure to identify data records that have been processed in the first replica;
- and
- resuming processing of the stream of data records between the first operator and the second operator via a second replica of the shard without processing the data records that have already been processed from the first replica.
Dependencies are inferred from the claim text.