# US 12,112,560 B2 · Usage based resource utilization of training pool for chatbots

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,560 B2
- Document ID: US12112560B2
- Status: Granted
- Kind: B2
- Application number: 17/570,804
- Filed: 2022-01-07
- Published: 2022-07-21
- Granted: 2024-10-08
- Earliest claimed: 2021-01-20
- Assignees: Oracle International Corporation
- Applicants: Oracle International Corporation
- Inventors: Xin Xu; Suman Mallapura Somasundar; Vishal Vishnoi; Xinwei Zhang; Ping L. Lin
- Web page: https://patentgrep.com/patent/US12112560B2

## CPC

- G06V 30/19113: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system}
- G06V 30/19147: {Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting}
- H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A training request including an identifier that is indicative of a type of a machine learning (ML) model that is to be trained is received. A plurality of workers are maintained in a training pool, and a plurality of jobs are maintained in a queue of training jobs. Each worker is configured to train a particular type of ML model. Upon the training request being validated, a training job is created for the request and submitted to the queue of training jobs. For each type of ML model, a first metric and a second metric is obtained. A target metric is computed based on the first and the second metrics. The number of workers included in the training pool is modified based on the target metric.

## Claims

### Claim 1

A method comprising:
receiving a training request, the training request including a first identifier indicative of a type of a machine learning model that is to be trained;
maintaining a plurality of training jobs in a queue of training jobs;
maintaining a plurality of workers in a training pool, wherein each worker is configured to train a particular type of machine learning model; and
responsive to the training request being validated:
creating a training job associated with the training request;
submitting the training job to the queue of training jobs;
obtaining, for each type of machine learning model, a first metric corresponding to a first number of the type of machine learning models included in the queue of training jobs;
obtaining, for each type of machine learning model, a second metric corresponding to a second number of workers included in the training pool, wherein the second number of workers are programmed to train the type of machine learning model;
computing a target metric based on the first metric and the second metric; and
modifying the second number of workers included in the training pool based on the target metric.

### Claim 2 (depends on 1)

The method of claim 1, wherein the training request includes a second identifier indicative of a version of the type of machine learning model that is to be trained, and a third identifier corresponding to a tenant identifier associated with a customer issuing the training request.

### Claim 3 (depends on 1)

The method of claim 1, wherein each worker of the plurality of workers in the training pool is further programmed to train a particular version of the type of machine learning model.

### Claim 4 (depends on 1)

The method of claim 1, wherein the target metric is computed as a ratio of the first metric to the second metric.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
maintaining, for each type of machine learning model, the second metric to have a value of at least K, wherein K corresponds to a minimum number of workers configured to train the type of machine learning model.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
storing the first metric and the second metric in a database in a first format;
converting, via an adapter, the first metric and the second metric in the first format to a first metric and second metric in a second format; and
computing the target metric based on the first metric and the second metric in the second format.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
receiving a first training request;
training the type of the machine learning model associated with the first training request;
receiving a second training request; and
training the type of the machine learning model associated with the second training request, wherein the first training request and the second training request are executed simultaneously.

### Claim 8 (depends on 2)

The method of claim 2, further comprising:
extracting the third identifier from the training request;
obtaining a threshold value associated with the third identifier, the threshold value corresponding to a maximum number of machine learning models that can be simultaneously trained for the customer; and
validating the training request based on a current number of machine learning models being trained for the customer being less than the threshold value.

### Claim 9 (depends on 4)

The method of claim 4, wherein modifying the second number of workers included in the training pool further comprises increasing or decreasing the second number of workers in the training pool such that the target metric achieves a value of one.

### Claim 10 (depends on 7)

The method of claim 7, wherein the first training request is issued by a first customer, and the second training request is issued by a second customer that is different than the first customer.

### Claim 11 (depends on 7)

The method of claim 7, wherein the first training request and the second training request are issued by a first customer.

### Claim 12

A non-transitory computer readable medium storing specific computer-executable instructions that, when executed by a processor, cause a computer system to at least:
receive a training request, the training request including a first identifier indicative of a type of a machine learning model that is to be trained;
maintaining a plurality of training jobs in a queue of training jobs;
maintain a plurality of workers in a training pool, wherein each worker is configured to train a particular type of machine learning model; and
in response to the training request being validated:
create a training job associated with the training request;
submit the training job to the queue of training jobs;
obtain, for each type of machine learning model, a first metric corresponding to a first number of the type of machine learning models included in the queue of training jobs;
obtain, for each type of machine learning model, a second metric corresponding to a second number of workers included in the training pool, wherein the second number of workers are programmed to train the type of machine learning model;
compute a target metric based on the first metric and the second metric; and
modify the second number of workers included in the training pool based on the target metric.

### Claim 13 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the training request includes a second identifier indicative of a version of the type of machine learning model that is to be trained, and a third identifier corresponding to a tenant identifier associated with a customer issuing the training request.

### Claim 14 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein each worker included in the training pool is further configured to train a particular version of the type of machine learning model.

### Claim 15 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the target metric is computed as a ratio of the first metric to the second metric.

### Claim 16 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the computer system is further configured to:
maintain, for each type of machine learning model, the second metric to have a value of at least K, wherein K corresponds to a minimum number of workers configured to train the type of machine learning model.

### Claim 17 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the computer system is further configured to:
store the first metric and the second metric in a database in a first format;
convert, via an adapter, the first metric and the second metric in the first format to a first metric and second metric in a second format; and
compute the target metric based on the first metric and the second metric in the second format.

### Claim 18 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the computer system is further configured to:
extract the third identifier from the training request;
obtain a threshold value associated with the third identifier, the threshold value corresponding to a maximum number of machine learning models that can be simultaneously trained for the customer; and
validate the training request based on a current number of machine learning models being trained for the customer being less than the threshold value.

### Claim 19 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the computer system is further configured to modify the second number of workers included in the training pool by increasing or decreasing the second number of workers in the training pool such that the target metric achieves a value of one.

### Claim 20

A system comprising:
a processor; and
a memory including instructions that, when executed with the processor, cause the system to, at least:
receive, by a training gateway, a training request including a first identifier indicative of a type of a machine learning model that is to be trained;
maintain a plurality of training jobs in a queue of training jobs;
maintain a plurality of workers in a training pool, wherein each worker is configured to train a particular type of machine learning model; and
in response to the training request being validated by the training gateway:
create a training job associated with the training request;
submit the training job to the queue of training jobs;
obtain, for each type of machine learning model, a first metric corresponding to a first number of the type of machine learning models included in the queue of training jobs;
obtain, for each type of machine learning model, a second metric corresponding to a second number of workers included in the training pool, wherein the second number of workers are programmed to train the type of machine learning model;
compute, by a horizontal pod auto-scaler (HPA), a target metric based on the first metric and the second metric; and
modify, by the HPA, the second number of workers included in the training pool based on the target metric.