Query set optimization in a data analytics pipeline

A computer-implemented method comprising:

• in a set of data analytics queries, at least a one of the queries comprising more than one operator, and each query being at least one of i) a producer of data for an other query in the set, and ii) a consumer of data from an other query in the set:
• identifying, by one or more computing devices, each producer/consumer relationship between the queries;
• identifying, by the one or more computing devices, one or more optimizations among the queries based on the identified relationships;
• and
• applying, by the one or more computing devices, at least one identified optimization to at least one of the queries.

A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

• in a set of data analytics queries, at least a one of the queries comprising more than one operator, and each query being at least one of i) a producer of data for an other query in the set, and ii) a consumer of data from an other query in the set:
• identifying each producer/consumer relationship between the queries;
• identifying one or more optimizations among the queries based on the identified relationships;
• and
• applying at least one identified optimization to at least one of the queries.