A non-transitory machine readable medium storing a program for managing a network connecting a plurality of graphics processing units (GPUs) that collectively execute a distributed application by performing operations for executing the distributed application, the program comprising sets of instructions for:
- configuring a plurality of forwarding elements that are part of a network to forward data message flows between the GPUs in order to allow the GPUs to share results of distributed-application operations that the GPUs execute;
- repeatedly collecting and analyzing telemetry data from the forwarding elements;
- and
- upon detecting, based on the telemetry analysis, performance degradation at a particular forwarding element, reconfiguring one or more forwarding elements to forward fewer data message flows through the particular forwarding element in order to mitigate the detected performance degradation.