Usage based resource utilization of training pool for chatbots

A non-transitory computer readable medium storing specific computer-executable instructions that, when executed by a processor, cause a computer system to at least:

• receive a training request, the training request including a first identifier indicative of a type of a machine learning model that is to be trained;
• maintaining a plurality of training jobs in a queue of training jobs;
• maintain a plurality of workers in a training pool, wherein each worker is configured to train a particular type of machine learning model;
• and
• in response to the training request being validated:
• create a training job associated with the training request;
• submit the training job to the queue of training jobs;
• obtain, for each type of machine learning model, a first metric corresponding to a first number of the type of machine learning models included in the queue of training jobs;
• obtain, for each type of machine learning model, a second metric corresponding to a second number of workers included in the training pool, wherein the second number of workers are programmed to train the type of machine learning model;
• compute a target metric based on the first metric and the second metric;
• and
• modify the second number of workers included in the training pool based on the target metric.

A system comprising:

• a processor;
• and
• a memory including instructions that, when executed with the processor, cause the system to, at least:
• receive, by a training gateway, a training request including a first identifier indicative of a type of a machine learning model that is to be trained;
• maintain a plurality of training jobs in a queue of training jobs;
• maintain a plurality of workers in a training pool, wherein each worker is configured to train a particular type of machine learning model;
• and
• in response to the training request being validated by the training gateway:
• create a training job associated with the training request;
• submit the training job to the queue of training jobs;
• obtain, for each type of machine learning model, a first metric corresponding to a first number of the type of machine learning models included in the queue of training jobs;
• obtain, for each type of machine learning model, a second metric corresponding to a second number of workers included in the training pool, wherein the second number of workers are programmed to train the type of machine learning model;
• compute, by a horizontal pod auto-scaler (HPA), a target metric based on the first metric and the second metric;
• and
• modify, by the HPA, the second number of workers included in the training pool based on the target metric.