Coflows for geo-distributed computer sites that communicate via wide area network

A computing device comprising a memory having computer readable instructions, and one or more processors for executing the computer readable instructions to configure the computing device to perform operations comprising:

• accessing a dependency graph structure that describes geo-distributed job objectives and data sources of a coflow, inter-task data volumes of the coflow, and one or more location-dependent constraints of the coflow;
• determining, for a stage of a plurality of successive computational stages, a plurality of geo-distributed computer sites based on the one or more location-dependent constraints;
• mapping the coflow to the plurality of geo-distributed computer sites, wherein
• the plurality of geo-distributed computer sites communicates via wide area network (WAN),
• the mapping is based on the to one or more location-dependent constraints, and
• the mapping includes placing a plurality of tasks at the plurality of geo-distributed computer sites;
• identifying multiple candidate data paths for each of a plurality of source-destination pairs of the plurality of geo-distributed computer sites;
• and
• performing a mathematical optimization to identify a set of paths from the multiple candidate data paths, wherein
• the identifying of the set of paths is based on a total flow completion time of the coflow and at least one additional objective of the coflow, and
• the at least one additional objective of the coflow is derived from the geo-distributed job objectives.

A computer program product comprising one or more computer-readable memory devices encoded with data including instructions that, when executed, causes a processor set to perform a method, comprising:

• accessing a dependency graph structure that describes geo-distributed job objectives and data sources of a coflow, inter-task data volumes of the coflow, and one or more location-dependent constraints of the coflow;
• determining, for a stage of a plurality of successive computational stages, a plurality of geo-distributed computer sites based on the one or more location-dependent constraints:
• mapping the coflow to the plurality of geo-distributed computer sites, wherein
• the plurality of geo-distributed computer sites communicates via wide area network (WAN),
• the mapping is based on one or more location-dependent constraints, and
• the mapping includes placing a plurality of tasks at the plurality of geo-distributed computer sites;
• identifying multiple candidate data paths for each of a plurality of source-destination pairs of the plurality of geo-distributed computer sites;
• and
• performing a mathematical optimization to identify a set of paths from the multiple candidate data paths, wherein
• the identifying of the set of paths is based on a total flow completion time of the coflow and at least one additional objective of the coflow, and
• the at least one additional objective of the coflow is derived from the geo-distributed job objectives.