Methods, systems, and computer readable media for testing the performance and reliability of a device or system under test (sut) using real and emulated processing ranks within a data center environment

A system for testing the performance and reliability of a device or system under test (SUT) using real and emulated processing ranks within a data center environment, the system comprising:

• a test system comprising a controller, at least one processor and a memory, and a connector for connecting to an electrical connector associated with a SUT comprising non-emulated processing units, and is configured to perform a test of the SUT comprising computer-executable instructions stored in the memory and executable by the at least one processor by instantiating a machine learning (ML)-framework-based plugin comprising an emulator configured for emulating processing units;
• communicating, from the controller on the test system, a configuration of the ML-framework-based plugin to non-emulated processing units comprising a collectives parameter indicating a quantity and rank information of the emulated processing units;
• performing the test of the SUT by:
• executing a ML workload on the non-emulated processing units;
• emulating execution of the ML workload on the emulated processing units;
• exchanging packets associated with the execution of the ML workload between the non-emulated processing units and the ML-framework-based plugin;
• and
• monitoring performance of the non-emulated processing units in executing the ML workload.

A non-transitory computer readable medium having stored thereon executable instructions that when executed by a processor of a computer control the computer to perform steps comprising:

• instantiating a machine learning (ML)-framework-based plugin comprising an emulator configured for emulating processing units;
• communicating, from a controller on a test system, a configuration of the ML-framework-based plugin to non-emulated processing units on a SUT comprising a collectives parameter indicating a quantity and rank information of the emulated processing units;
• performing a test of the SUT by:
• executing a ML workload on the non-emulated processing units;
• emulating execution of the ML workload on the emulated processing units;
• exchanging packets associated with the execution of the ML workload between the non-emulated processing units and the ML-framework-based plugin;
• and
• monitoring performance of the non-emulated processing units in executing the ML workload.