Fair simultaneous comparison of parallel machine learning models

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:

• receive a plurality of machine learning (ML) algorithms;
• receive a total amount of hardware resources;
• and
• execute each of the plurality of ML algorithms using a respective percentage of the total amount of hardware resources, wherein:
• input for each of the plurality of ML algorithms comprises a resource preference profile that includes a plurality of hardware relative utility metrics,
• the plurality of hardware relative utility metrics is based on a resource allocation index, a resource distribution index, and a resource variance index,
• bias stemming from different hardware requirements or preferences for each of the plurality of ML algorithms is eliminated or mitigated,
• the respective performance of each of the plurality of ML algorithms associated with the execution is based on at least one of the plurality of hardware relative utility metrics describing a hardware relative utility of any given resource allocation for each of the plurality of ML algorithms, and
• a particular ML solution is based on the plurality of ML algorithms.

An apparatus comprising:

• a memory configured to store instructions;
• and
• a processor configured to execute the instructions to:
• receive a plurality of machine learning (ML) algorithms;
• receive a total amount of hardware resources;
• and
• execute each of the plurality of ML algorithms using a respective percentage of the total amount of hardware resources, wherein:
• input for each of the plurality of ML algorithms comprises a resource preference profile that includes a plurality of hardware relative utility metrics,
• the plurality of hardware relative utility metrics is based on a resource allocation index, a resource distribution index, and a resource variance index,
• bias stemming from different hardware requirements or preferences for each of the plurality of ML algorithms is eliminated or mitigated,
• the respective performance of each of the plurality of ML algorithms associated with the execution is based on at least one of the plurality of hardware relative utility metrics describing a hardware relative utility of any given resource allocation for each of the plurality of ML algorithms, and
• a particular ML solution is based on the plurality of ML algorithms.