Streaming system performance optimization

A non-transitory computer readable medium including machine readable instructions executable by a processor to:

• receive a dataflow directed acyclic graph comprising nodes and edges connecting the nodes, wherein the nodes comprise producers and consumers processing tuples, and the edges send the tuples from the producers to the consumers;
• calculate parameters for the nodes to minimize average latencies for the tuples, wherein the parameters include batch size and a degree of parallelism (dop);
• and
• send the parameters to a streaming system to process the dataflow according to the parameters.

A method comprising:

• receiving a dataflow directed acyclic graph comprising nodes and edges connecting the nodes, wherein the nodes comprise producers and consumers processing tuples, and the edges send the tuples from the producers to the consumers;
• determining, by a processor, for each producer, whether batching is to be performed to reduce actual processing time, and a batch size if the batching is to be performed;
• and
• determining a dop for each of the consumers based on an output rate of tuples sent from a producer to the consumer and a pure processing time of the tuples at the consumer.