System and method for dynamic activation of real-time streaming data overflow paths

A computer program product comprising a non-transitory computer readable storage medium having a plurality of instructions stored thereon, which, when executed by a processor, cause the processor to perform operations including:

• defining one or more potential overflow paths for processing at least a portion of a data load through a plurality of stream operators deployed on a plurality of computing devices, wherein each stream operator of the plurality of stream operators is configured to perform one or more logical operations on the at least a portion of the data load as the at least a portion of the data load passes from one stream operator to a next stream operator in the one or more potential flow paths;
• determining a conditional availability for each computing device of the plurality of computing devices;
• identifying a change in one or more of a configuration of the plurality of computing devices and the at least a portion of the data load;
• predicting, on at least one processor, a future change in one or more of the configuration of the plurality of computing devices and the at least a portion of the data load based upon, at least in part, the conditional availability for each computing device of the plurality of computing devices and observed historical application patterns;
• dynamically activating, on the at least one processor, one or more potential overflow paths in response to the identified change based upon, at least in part, the conditional availability for each computing device of the plurality of computing devices and a tuple rate load balancing, the tuple rate load balancing designating an order of the data load, wherein the one or more potential overflow paths includes at least one duplicate stream operator based on the plurality of stream operators;
• and
• dynamically activating, on the at least one processor, the one or more potential overflow paths in anticipation of the predicted future change, wherein the one or more potential overflow paths includes the at least one duplicate stream operator based on the plurality of stream operators, and wherein dynamically activating the one or more potential overflow paths in anticipation of the predicted future change includes creating dynamic connections to the at least one duplicate stream operator.

A computing system comprising:

• a processor and a memory module coupled with the processor, the processor being configured for:
• defining one or more potential overflow paths for processing at least a portion of a data load through a plurality of stream operators deployed on a plurality of computing devices, wherein each stream operator of the plurality of stream operators is configured to perform one or more logical operations on the at least a portion of the data load as the at least a portion of the data load passes from one stream operator to a next stream operator in the one or more potential flow paths;
• determining a conditional availability for each computing device of the plurality of computing devices;
• identifying a change in one or more of a configuration of the plurality of computing devices and the at least a portion of the data load;
• predicting, on at least one processor, a future change in one or more of the configuration of the plurality of computing devices and the at least a portion of the data load based upon, at least in part, the conditional availability for each computing device of the plurality of computing devices and observed historical application patterns;
• dynamically activating, on the at least one processor, one or more potential overflow paths in response to the identified change based upon, at least in part, the conditional availability for each computing device of the plurality of computing devices and a tuple rate load balancing, the tuple rate load balancing designating an order of the data load, wherein the one or more potential overflow paths includes at least one duplicate stream operator based on the plurality of stream operators;
• and
• dynamically activating, on the at least one processor, the one or more potential overflow paths in anticipation of the predicted future change, wherein the one or more potential overflow paths includes the at least one duplicate stream operator based on the plurality of stream operators, and wherein dynamically activating the one or more potential overflow paths in anticipation of the predicted future change includes creating dynamic connections to the at least one duplicate stream operator.