# US 2025/0094840 A1 · Inference model selection method considering task request rate

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094840 A1
- Document ID: US20250094840A1
- Status: Published application
- Kind: A1
- Application number: 18/443,445
- Filed: 2024-02-16
- Published: 2025-03-20
- Earliest claimed: 2023-09-14
- Assignees: Korea University Research and Business Foundation
- Applicants: Korea University Research and Business Foundation
- Inventors: Sangheon PACK; Haneul KO; Hyeonjae JEONG
- Web page: https://patentgrep.com/patent/US20250094840A1

## CPC

- G06N 5/04: Inference or reasoning models
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed is an inference model selection method and apparatus considering a task request rate. The inference model selection method is performed by a computing device including a processor and includes monitoring computing resources of the computing device; receiving a task; selecting an inference model to perform inference for the received task; inputting the task into a queue of the selected inference model; and performing an inference operation.

## Claims

### Claim 1

An inference model selection method performed by a computing device comprising a processor, the interference model selection method comprising:
monitoring computing resources of the computing device;
receiving a task;
selecting an inference model to perform inference for the received task;
inputting the task into a queue of the selected inference model; and
performing an inference operation.

### Claim 2 (depends on 1)

The inference model selection method of claim 1, wherein the monitoring is periodically performed at predetermined periods.

### Claim 3 (depends on 2)

The inference model selection method of claim 2, wherein the selecting of the inference model comprises selecting an appropriate inference model for tasks received over the predetermined period, and
the inference model includes at least one Single-task Learning (STL) model and at least one Multi-task Learning (MTL) model.

### Claim 4 (depends on 3)

The inference model selection method of claim 3, wherein the selecting of the inference model comprises solving a Q-function derived as a result of transforming a Markov Decision Process (MDP) model to Q-Learning, and
the Q-function is represented as Q,(S,A)=maxπ[Qπ(S,A)],
where S denotes a state space, A denotes an action space, and π denotes a policy.

### Claim 5 (depends on 4)

The inference model selection method of claim 4, wherein the performing of the inference operation comprises performing the inference operation by inputting a task input to a queue to the STL model without a waiting time if the STL model is selected.

### Claim 6 (depends on 5)

The inference model selection method of claim 5, wherein the performing of the inference operation comprises performing the inference operation by, when the queue is full of tasks, inputting the task input to the queue to the MTL model if the MTL model is selected.

### Claim 7 (depends on 5)

The inference model selection method of claim 5, wherein the performing of the inference operation comprises performing the inference operation by, when a sum of a time elapsed from a point in time at which at least one task stored in the queue is received and a computing time required for inference is equal to a value acquired by subtracting a predetermined period of time from a required delay time, inputting all the tasks stored in the queue to the MTL model, if the MTL model is selected.

### Claim 8 (depends on 5)

The inference model selection method of claim 5, wherein the performing of the inference operation comprises performing the inference operation by, when a sum of a waiting time of a task in the queue and a computing time required for inference is equal to a value acquired by subtracting a predetermined period of time from a required delay time, inputting all the tasks stored in the queue to the MTL model, if the MTL model is selected.

### Claim 9 (depends on 8)

The inference model selection method of claim 8, further comprising:
transmitting an inference result.
