Utilizing top of rack switch caching for executing artificial intelligence workloads

A physical server, comprising:

• a plurality of graphics processing units (GPUs),
• a plurality of data processing units (DPUs),
• a peripheral connection interface express (PCIe) fabric connecting the plurality of GPUs and the plurality of DPUs,
• wherein each of the plurality of GPUs is configured to obtain checkpoint data associated with an artificial intelligence (AI) workload and transmit, via one of the plurality of DPUs, the checkpoint data to a corresponding mapped cache in a Top of Rack (ToR) switch,
• wherein the ToR Switch is configured to transmit the checkpoint data via a scale out network to a storage system that is external to the physical server and the ToR Switch.

A system comprising:

• a physical server, comprising:
• a plurality of graphics processing units (GPUs),
• a plurality of data processing units (DPUs),
• a peripheral connection interface express (PCIe) fabric connecting the plurality of GPUs and the plurality of DPUs,
• wherein each of the plurality of GPUs is configured to obtain checkpoint data associated with an artificial intelligence (AI) workload and transmit, via one of the plurality of DPUs, the checkpoint data to a corresponding mapped cache in a Top of Rack (ToR) switch, and
• the ToR Switch is configured to transmit the checkpoint data via a scale out network to a storage system that is external to the physical server and the ToR Switch,
• wherein the ToR Switch and the physical server are in a server rack.