Load testing and performance benchmarking for large language models using a cloud computing platform

A system for evaluating a performance of an artificial intelligence model for a cloud computing platform, the system comprising:

• a processing system;
• a computer-readable medium having encoded thereon, instructions that when executed by the processing system causes the system to:
• select a load profile generated to represent a workload context by defining at least a number of containers and a number of threads per container;
• generate a representative workload configured to evaluate the performance of the artificial intelligence model for the cloud computing platform under the workload context represented by the load profile;
• execute the representative workload on a computing resource utilizing the artificial intelligence model;
• extract a plurality of performance metrics from the from the computing resource following execution of the representative workload on the computing resource utilizing the artificial intelligence model;
• synthesize a performance measurement based on the plurality of performance metrics extracted from the computing resource;
• update the load profile using the performance measurement by modifying the number of containers and the number of threads per container defined by the load profile;
• and
• generate, utilizing the modified number of containers and the modified number of threads per container defined by the updated load profile, an updated representative workload configured to further evaluate the performance of the artificial intelligence model for the cloud computing platform under a different workload context.

A computer-readable storage medium having encoded thereon computer-readable instructions that when executed by a processing system causes a system to:

• select a load profile generated to represent a workload context by defining at least a number of containers and a number of threads per container;
• generate a representative workload configured to evaluate a performance of an artificial intelligence model for a cloud computing platform under the workload context represented by the load profile;
• execute the representative workload on a computing resource utilizing the artificial intelligence model;
• extract a plurality of performance metrics from the computing resource following execution of the representative workload on the computing resource utilizing the artificial intelligence model;
• synthesize a performance measurement based on the plurality of performance metrics extracted from the computing resource;
• update the load profile using the performance measurement by modifying the number of containers and the number of threads per container defined by the load profile;
• and
• generate, utilizing the modified number of containers and the modified number of threads per container defined by the updated load profile, an updated representative workload configured to further evaluate the performance of the artificial intelligence model for the cloud computing platform under a different workload context.