Parallel computing of ml services and applications

A data processing system, comprising:

• a processor;
• and
• a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including:
• obtaining a performance profile corresponding to times taken to perform an inferencing by a machine learning (ML) model using a different number of processing resources from a plurality of processing resources;
• determining one or more groupings of processing resources from the plurality of processing resources, each grouping includes one or more partitions
• calculating performance speeds corresponding to each grouping based on the performance profile;
• determining a grouping having a best performance speed from the calculated performance speeds;
• and
• partitioning the plurality of processing resources based on the determined grouping to perform the inferencing.

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

• obtaining a performance profile corresponding to times taken to perform an inferencing by a machine learning (ML) model using a different number of processing resources from a plurality of processing resources;
• determining one or more groupings of processing resources from the plurality of processing resources, each grouping includes one or more partitions;
• calculating performance speeds corresponding to each grouping based on the performance profile;
• determining a grouping having a best performance speed from the calculated performance speeds;
• and
• partitioning the plurality of processing resources based on the determined grouping to perform the inferencing.