Predictive operator graph element processing

A computer program product for predictively starting a processing element, comprising:

• a computer-readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:
• computer readable program code to receive streaming data to be processed by a plurality of operators, wherein each of the plurality of operators is configured to perform a respective one or more operations on received units of data;
• computer readable program code to establish an operator graph through the plurality of operators that defines at least one execution path through the plurality of operators, and wherein at least one of the plurality of operators is configured to receive data from at least one upstream operator and transmit data to at least one downstream operator;
• computer readable program code to determine a respective historical startup time for each of two or more operators in the operator graph, which, once started, begin performing the respective one or more operations upon receiving a respective requisite amount of data from a respective one or more upstream operators;
• computer readable program code to determine, for each of the two or more operators, a respective estimated amount of time the respective operator takes to receive the requisite amount of data from the respective one or more upstream operators, based on a rate at which data was historically received from the respective one or more upstream operators;
• and
• computer readable program code to predictively start each of the two or more operators at a respective startup time prior to the respective historical startup time based on the respective estimated amount of time historically taken to receive the requisite amount of data, such that the operator receives the requisite amount of data for performing the respective one or more operators by the determined historical startup time.

A system, comprising:

• a processor;
• and
• a memory containing a program that, when executed by the processor, performs an operation for predictively starting a processing element, comprising:
• receiving streaming data to be processed by a plurality of operators, each configured to perform a respective one or more operations on received units of data;
• establishing an operator graph through the plurality of operators that defines an execution path through the plurality of operators, and wherein at least one of the plurality of operators is configured to receive data from at least one upstream operator and transmit data to at least one downstream operator;
• determining historical startup times for two or more operators in the operator graph, wherein the two or more operators, once started, begin performing the respective one or more operations upon receiving a respective requisite amount of data from respective one or more upstream operators;
• determining, for each of the two or more operators, a respective estimated amount of time the operator takes to receive the requisite amount of data from the one or more upstream operators, based on a size of the requisite amount of data and a rate at which data was historically received at the operator from the one or more upstream operators;
• and
• for each of the two or more operators, predictively starting the operator at a respective startup time prior to the determined historical startup time for the operator, based on the determined amount of time historically taken to receive the requisite amount of data, such that the first operator receives the requisite amount of data for performing the respective one or more operators by the determined historical startup time.