# US 2024/0187312 A1 · Adaptive batch processing method and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0187312 A1
- Document ID: US20240187312A1
- Status: Published application
- Kind: A1
- Application number: 18/443,680
- Filed: 2024-02-16
- Published: 2024-06-06
- Earliest claimed: 2021-09-03
- Assignees: Korea Electronics Technology Institute
- Applicants: Korea Electronics Technology Institute
- Inventors: Seung Woo KUM; Jae Won MOON; Seung Taek OH
- Web page: https://patentgrep.com/patent/US20240187312A1

## CPC

- H04L 41/16: using machine learning or artificial intelligence
- G06F 9/50: Allocation of resources, e.g. of the central processing unit [CPU]
- G06F 9/54: Interprogram communication
- G06N 20/00: Machine learning
- H04L 65/40: Support for services or applications
- H04L 67/61: taking into account QoS or priority requirements
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An adaptive batch processing and apparatus is proposed. The adaptive batch processing method may be performed by a system including a server which saves a model trained by machine learning and a client which transmits input data by connecting to the server while requesting an analysis result based on the model for the corresponding input data. The method may include measuring an amount of available resources of the server in the server, calculating a batch size using the amount of the available resources measured by the server, and transmitting information about the calculated batch size to the client. The method may further include generating batch data of a size reflecting the transmitted information by generating a queue of a plurality of input data at the client, transmitting the generated batch data to the server, and performing an analysis based on the model on the batch data received from the server.

## Claims

### Claim 1

A method performed by a system including a server which saves a model trained by machine learning and a client which transmits input data by connecting to the server while requesting an analysis result based on the model for the corresponding input data, respectively, the method comprising:
measuring, by the server, an amount of available resources of the server;
calculating a batch size using the amount of available resources measured by the server, and transmitting information about the calculated batch size to the client;
generating batch data of a size reflecting the transmitted information by generating a queue of a plurality of input data in the client, and transmitting the generated batch data to the server; and
performing analysis based on the model on the batch data received from the server.

### Claim 2 (depends on 1)

The method of claim 1, wherein the amount of available resources includes a processor load size and a free size of memory of the server.

### Claim 3 (depends on 1)

The method of claim 1, wherein, when generating the batch data, in response to a collection time of the plurality of input data corresponding to the batch size exceeding a delay time of quality of service (QoS) required for the analysis, batch data of a size smaller than the batch size is generated.

### Claim 4 (depends on 1)

The method of claim 1, wherein, when calculating the batch size, the batch size is calculated using the amount of available resources of the server and an average size of input data previously transmitted from the client to the server, respectively.

### Claim 5

A method performed by a system including a server which saves a model trained by machine learning and a client which transmits input data by connecting to the server while requesting an analysis result based on the model for the corresponding input data, respectively, the method comprising:
measuring, by the server, an amount of available resources of the server and transmitting the measured amount of available resources to the client;
calculating a batch size using the amount of available resources transmitted from the client;
generating batch data of a size reflecting the batch size by generating a queue of a plurality of input data in the client and transmitting the generated batch data to the server; and
performing analysis based on the model on the batch data received from the server.

### Claim 6 (depends on 5)

The method of claim 5, wherein the amount of available resources includes a processor load size and a free size of memory of the server.

### Claim 7 (depends on 5)

The method of claim 5, wherein, when generating the batch data, in response to a collection time of the plurality of input data corresponding to the batch size exceeding a delay time of quality of service (QoS) required for the analysis, batch data of a size smaller than the batch size is generated.

### Claim 8 (depends on 5)

The method of claim 5, wherein, when calculating the batch size, the batch size is calculated using the amount of available resources of the server and an average size of input data previously transmitted from the client to the server, respectively.

### Claim 9

A system including a server which saves a model trained by machine learning and a client which transmits input data by connecting to the server while requesting an analysis result based on the model for the corresponding input data, the system comprising:
a resource amount measurement processor configured to measure an amount of available resources of the server;
a batch size calculation processor configured to calculate a batch size using the amount of available resources;
a queue generation processor included in the client and configured to generate a queue of a plurality of input data in the client;
a batch generation processor included in the client and configured to generate batch data reflecting the batch size in the generated queue; and
a model analysis processor included in the server and configured to perform analysis on the batch data received from the client based on the model.

### Claim 10 (depends on 9)

The system of claim 9, wherein the available resource includes a processor load size and a free size of memory of the server.

### Claim 11 (depends on 9)

The system of claim 9, wherein, in response to a collection time of the plurality of input data corresponding to the batch size exceeding a delay time of quality of service (QoS) required for the analysis, the queue generation processor is configured to generate batch data of a size smaller than the batch size.

### Claim 12 (depends on 9)

The system of claim 9, wherein the batch size calculation processor is configured to calculate the batch size using the amount of available resources of the server and an average size of input data previously transmitted from the client to the server, respectively.
