Identifying and routing poison tuples in a streaming application
AssigneeInternational Business Machines Corporation
InventorsMichael J. Branson, John M. Santosuosso
Life of this patent
Estimating the term…
Abstract
A method for processing a stream of tuples may comprise receiving a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. In addition, the method may include generating a model of performance for processing the stream of tuples at runtime, wherein one or more tuples from the stream of tuples potentially cause adverse performance. Further, the method may comprise predicting a parameter for a tuple from the stream of tuples, the parameter indicating a potential for adverse performance, the predicting including using the model. The method may also include modifying processing of the tuple if the parameter falls outside a threshold.
Claims20
A computer program product for processing a stream of tuples, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code readable by a computer to cause the computer to:
- receive a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors, wherein one or more poison tuples from the stream of tuples has one or more attributes with values that cause adverse performance in a processing element;
- generate a model of performance for processing the stream of tuples at runtime;
- predict a parameter of the one or more poison tuples from the stream of tuples indicating a potential of a processing element for adverse performance when processing one or more poison tuples, the predicting including using the model;
- and
- modify processing of the one or more poison tuples if the parameter falls outside a parameter threshold.
A system for processing a stream of tuples, comprising:
- a plurality of processing elements configured to receive a stream of tuples, wherein one or more poison tuples from the stream of tuples has one or more attributes with values that cause adverse performance in a processing element, the processing elements operating on one or more computer processors, each processing element having one or more stream operators;
- a tuple management system (TMS) configured to generate a prediction for a poison tuple from the one or more poison tuples, wherein the prediction indicates a potential of one or more stream operators for adverse performance when processing the poison tuple, wherein the TMS uses a model of performance;
- and
- a first stream operator, configured to:
- receive the prediction and a parameter threshold,
- compare the prediction to the parameter threshold, and
- route the poison tuple to a second stream operator if the prediction falls outside the parameter threshold, wherein the second stream operator is included in a bypass execution path.
Dependencies are inferred from the claim text.