# US 2026/0229139 A1 · Operating method for configuring model performing reinforcement learning related to task planning and electronic apparatus supporting thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0229139 A1
- Document ID: US20260229139A1
- Status: Published application
- Kind: A1
- Application number: 19/233,189
- Filed: 2025-06-10
- Published: 2026-08-06
- Earliest claimed: 2025-02-06
- Assignees: Agency for Defense Development
- Applicants: Agency for Defense Development
- Inventors: Sun Ju LEE; Do Hyung KIM; Beom Ki KIM
- Web page: https://patentgrep.com/patent/US20260229139A1

## CPC

- G09B 9/003: {for military purposes and tactics}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/096: Transfer learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Provided is a method of configuring a model by an electronic apparatus, the method including designing a multi-agent reinforcement learning structure to perform reinforcement learning based on each agent configured to correspond to each military unit identified as a target of task planning in a virtual battlespace simulation, and configuring a model performing reinforcement learning related to task planning for each military unit based on the multi-agent reinforcement learning structure.

## Claims

### Claim 1

A method of configuring a model by an electronic apparatus, the method comprising:
designing a multi-agent reinforcement learning structure to perform reinforcement learning based on each agent configured to correspond to each military unit identified as a target of task planning in a virtual battlespace simulation; and
configuring a model performing reinforcement learning related to task planning for each military unit based on the multi-agent reinforcement learning structure,
wherein the multi-agent reinforcement learning structure is designed to include each agent reinforcement learning structure configured to correspond to each agent in order that each agent learns the task planning for each military unit,
wherein each agent reinforcement learning structure comprises:
an attention network structure generating battlespace information reflecting an importance of each military unit based on information about each military unit identified through the virtual battlespace simulation; and
a task selection network structure determining and selecting a task to be performed by each military unit based on the battlespace information among one or more tasks performable by each military unit in the virtual battlespace simulation,
wherein the task selection network structure includes a selection network selecting the task to be performed by each military unit among the one or more tasks and one or more task networks respectively corresponding to the one or more tasks, and
wherein when a specific task is selected among the one or more tasks by the selection network, a specific task network corresponding to the specific task among one or more task networks is activated and the activated specific task network is set to output input value information for performing the specific task.

### Claim 2 (depends on 1)

The method of claim 1, wherein an embedding vector including the information about each military unit is input into the attention network structure,
wherein the embedding vector including the information about each military unit is reconfigured to an embedding vector of each military unit and an embedding vector corresponding to a characteristic of each agent,
wherein, based on the embedding vector of each military unit and the embedding vector corresponding to the characteristic of each agent, each unit battlespace information determined with regard to each military unit and each weight assigned to each military unit with respect to each agent in the virtual battlespace simulation are calculated, and
wherein the battlespace information reflecting the information about each military unit is generated based on an operation over each unit battlespace information and each weight.

### Claim 3

(canceled)

### Claim 4 (depends on 1)

The method of claim 1, wherein each agent is configured to learn the task planning for each military unit based on a single-agent reinforcement learning algorithm.

### Claim 5 (depends on 1)

The method of claim 1, wherein a learning scenario for the virtual battlespace simulation is configured to be automatically generated.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
beginning the virtual battlespace simulation according to a learning scenario based on a battlespace environment simulator;
in response to the beginning of the virtual battlespace simulation, obtaining each initial observation information identified with regard to each military unit from the battlespace environment simulator; and
obtaining information, which is output by the model corresponding to each initial observation information, about the task to be performed by each military unit.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
inputting the information about the task to be performed by each military unit into the battlespace environment simulator;
in response to a result of conducting the virtual battlespace simulation on the battlespace environment simulator based on inputting the information about the task to be performed by each military unit, identifying each observation information identified with regard to each military unit and each reward information about each agent according to each observation information; and
updating the model based on each observation information and each reward information.

### Claim 8 (depends on 7)

The method of claim 7, wherein updating the model is performed until an episode termination condition of the virtual battlespace simulation and a learning termination condition of the reinforcement learning for the task planning of each military unit are satisfied.

### Claim 9

A non-transitory computer-readable storage medium having a program for executing a method of configuring a model on a computer, the method comprising:
designing a multi-agent reinforcement learning structure to perform reinforcement learning based on each agent configured to correspond to each military unit identified as a target of task planning in a virtual battlespace simulation; and
configuring a model performing reinforcement learning related to task planning for each military unit based on the multi-agent reinforcement learning structure,
wherein the multi-agent reinforcement learning structure is designed to include each agent reinforcement learning structure configured to correspond to each agent in order that each agent learns the task planning for each military unit, and
wherein each agent reinforcement learning structure comprises:
importance of each military unit based on information about each military unit identified through the virtual battlespace simulation; and
a task selection network structure determining and selecting a task to be performed by each military unit based on the battlespace information among one or more tasks performable by each military unit in the virtual battlespace simulation,
wherein the task selection network structure includes a selection network selecting the task to be performed by each military unit among the one or more tasks and one or more task networks respectively corresponding to the one or more tasks, and
wherein when a specific task is selected among the one or more tasks by the selection network, a specific task network corresponding to the specific task among one or more task networks is activated and the activated specific task network is set to output input value information for performing the specific task.

### Claim 10

An electronic apparatus configuring a model, the electronic apparatus comprising:
a processor; and
one or more memories configured to store one or more instructions,
wherein the one or more instructions, when executed, control the processor to perform:
designing a multi-agent reinforcement learning structure to perform reinforcement learning based on each agent configured to correspond to each military unit identified as a target of task planning in a virtual battlespace simulation; and
configuring a model performing reinforcement learning related to task planning for each military unit based on the multi-agent reinforcement learning structure,
wherein the multi-agent reinforcement learning structure is designed to include each agent reinforcement learning structure configured to correspond to each agent in order that each agent learns the task planning for each military unit, and
wherein each agent reinforcement learning structure comprises:
importance of each military unit based on information about each military unit identified through the virtual battlespace simulation; and
a task selection network structure determining and selecting a task to be performed by each military unit based on the battlespace information among one or more tasks performable by each military unit in the virtual battlespace simulation,
wherein the task selection network structure includes a selection network selecting the task to be performed by each military unit among the one or more tasks and one or more task networks respectively corresponding to the one or more tasks, and
wherein when a specific task is selected among the one or more tasks by the selection network, a specific task network corresponding to the specific task among one or more task networks is activated and the activated specific task network is set to output input value information for performing the specific task.
