US 12,591,784 B2Granted

Decision-making method for agent action and related device

AssigneeHuawei Technologies Co., Ltd.

InventorsGongzheng Zhang, Bin Hu, Chen Xu, Jian Wang, Rong Li

Life of this patent

Estimating the term…

Loading patent…

Abstract

A decision-making method for an agent action and a related device are provided and are used in the field of communication technologies. The method includes: a first agent processes first state information obtained from an environment through a first model, to obtain a first cooperation message; the first agent sends the first cooperation message to at least one second agent; the first agent receives second cooperation message sent by the at least one second agent; the first agent processes the first cooperation message and the second cooperation message through a second model, to obtain a first cooperation action performed by the first agent, where the second cooperation message is sent by the at least one second agent.

Claims15

Dependent claims

A decision-making method for an agent action, comprising:

processing, by a first agent through a first model, said first model comprising a first neural network, first state information obtained from an environment, to obtain a first cooperation message, and sending the first cooperation message from the first agent to at least one second agent;
and
processing, by the first agent through a second model, said second model comprising a second neural network, the first cooperation message and a second cooperation message, to obtain a first cooperation action that needs to be performed by the first agent, and the second cooperation message is sent by the at least one second agent and received by the first agent, wherein
the decision-making method further comprises continuously optimizing the first model and the second model via steps of reward-based evaluation and training, the steps of reward-based evaluation and training comprising at least initializing the first model with a first weight matrix and first offset vector and initializing the second model with a second weight matrix and second offset vector, and then performing a number of steps equal to a predetermined number of iterations, wherein each step after a first step comprises altering one or more of the first weight matrix, first offset vector, second weight matrix, and second offset vector based on a fed back reward from a previous step, and evaluating, for said step after the first step, a new first cooperation message and a new second cooperation message to determine a new reward;
the first model and the second model are determined based on a same reward;
the first cooperation message is further used to determine a second cooperation action that needs to be performed by the at least one second agent;
the reward is related to state information, the first cooperation message, the second cooperation action, and the first cooperation action;
and
the state information comprises the first state information.

6 more elements

A first agent comprising:

a processing unit, configured to process, through a first model comprising a first neural network, first state information obtained from an environment, to obtain a first cooperation message, and
a sending unit, configured to send the first cooperation message from the first agent to at least one second agent;
the processing unit is further configured to process the first cooperation message and a second cooperation message through a second model comprising a second neural network, to obtain a first cooperation action performed by the first agent, and the second cooperation message is sent by the at least one second agent and received by the first agent;
wherein the processing unit is further configured to continuously optimize the first model and the second model via steps of reward-based evaluation and training, the steps of reward-based evaluation and training comprising at least initializing the first model with a first weight matrix and first offset vector and initializing the second model with a second weight matrix and second offset vector, and then performing a number of steps equal to a predetermined number of iterations, wherein each step after a first step comprises altering one or more of the first weight matrix, first offset vector, second weight matrix, and second offset vector based on a fed back reward from a previous step, and evaluating, for said step after the first step, a new first cooperation message and a new second cooperation message to determine a new reward;
wherein the first model and the second model are determined based on a same reward;
wherein the first cooperation message is further used to determine a second cooperation action that needs to be performed by the at least one second agent;
wherein the reward is related to state information, the first cooperation message, the second cooperation action, and the first cooperation action;
and
wherein the state information comprises the first state information.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/964,233Filed 12 Oct 2022 · continuation
Priority: CN 202010306503.717 Apr 2020
Published: US 2023/0032176 A12 Feb 2023
Granted: 31 Mar 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification9 codes

G05B 13/027
The criterion being a learning criterion: using neural networks only
G06N 3/092
Learning methods: reinforcement learning
G06N 3/006
Artificial life, i.e. computing arrangements simulating life: based on simulated virtual individual or collective life forms, e.g. social…
G06N 3/084
Learning methods: backpropagation, e.g. using gradient descent