Diffusing packets to identify faulty network apparatuses in multipath inter-data center networks

A data center comprising:

• a multipath network fabric including a multi-layer set of redundant routers;
• multiple racks each comprising multiple machines, wherein at least a subset of the machines are configured to:
• identify a set of target machines in one or more target data centers, wherein each target machine of the set of target machines has one or more available destination ports and each source machine of the one or more source machines has one or more available source ports;
• determine, based on the available source ports and the available destination ports, each available source port/destination port combination between the source data center and the one or more target data centers;
• diffuse packets through various paths of the multipath fabric to collectively identify a faulty network apparatus, wherein diffusing the packets comprises sending out bursts of packets with varying packet header parameters to each available source port/destination port combination,
• issue tracerts for each packet drop from the bursts of packets to identify corresponding failed routes,
• consolidate the failed routes from the subset of the machines, and
• triangulate the consolidated failed routes to identify the faulty network apparatus.

An apparatus comprising:

• one or more computer readable storage media;
• and
• program instructions stored on the one or more computer readable storage media which, when executed by one or more processing systems, direct the one or more processing systems to at least:
• identify a set of target machines in one or more target data centers, wherein each target machine of the set of target machines has one or more available destination ports and each source machine of the one or more source machines has one or more available source ports;
• determine, based on the available source ports and the available destination ports, each available source port/destination port combination between the source data center and the one or more target data centers;
• diffuse packets through various paths of a multipath inter-data center network fabric, wherein to diffuse the packets, the one or more processing systems sends out bursts of packets with varying packet header parameters to each available source port/destination port combination;
• monitor packet drops from the bursts of packets;
• issue tracerts for each of the packet drops to identify corresponding failed routes;
• and
• triangulate the failed routes to identify a faulty network apparatus in a source data center.