Partial fault tolerant stream processing applications

An article of manufacture, comprising:

• a computer readable storage medium containing an executable program for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators, where the program performs steps comprising:
• defining a quality score function that expresses how well the application is performing quantitatively;
• injecting a fault into at least one of the plurality of operators;
• assessing an impact of the fault on the quality score function;
• and
• selecting at least one partial fault-tolerant technique for implementation in the application based on the assessing.

A system for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators, the system comprising:

• a pre-processor that receives as inputs:
• source code for the application, a target stream operator that is one of the plurality of stream operators and at which a fault is to be injected, a fault model corresponding to a partial fault tolerance technique, and a plurality of injection parameters for the partial fault tolerance technique, wherein the pre-processor produces as output a plurality of instances of modified source code for the application, each of the plurality of instances of modified source code incorporating a different one of the plurality of injection parameters;
• and
• an application compiler coupled to the output of the pre-processor, where the application compiler produces as output executable code for each of the plurality of instances of modified source code.

A method for injecting a fault into a stream operator in a data stream processing application, the method comprising:

• developing a fault injection operator having specialized logic to emulate an effect of a failure, the fault injection operator corresponding to a fault model;
• embedding the fault injection operator into a data flow graph for the data stream processing application, so as to emulate at least one possible effect of the fault model on the data stream processing application,
• wherein the fault injection operator is only embedded only in one or more positions of the data flow graph at which the fault model manifests, and
• wherein the fault injection operator emulates a different failure when embedded at each of the one or more positions in the data flow graph.