# US 2024/0189718 A1 · Game character behavior control method and apparatus, storage medium, and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0189718 A1
- Document ID: US20240189718A1
- Status: Published application
- Kind: A1
- Application number: 18/582,558
- Filed: 2024-02-20
- Published: 2024-06-13
- Earliest claimed: 2020-04-28
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Weibin QIU
- Web page: https://patentgrep.com/patent/US20240189718A1

## CPC

- A63F 13/57: Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game (A63F13/56 takes precedence)
- A63F 13/35: Details of game servers
- A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/092: Reinforcement learning
- A63F 13/56: Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
- A63F 2300/65: for computing the condition of a game character
- A63F 2300/6623: for animating a group of characters
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure provides a game character behavior control method. The method includes obtaining current game state information of a game, and performing feature extraction on the current game state information to obtain first feature information that reflects parameters of game characters of opposing parties in the game; constructing second feature information associated with a target game character, the second feature information reflecting the first feature information and features of target state information associated with target game character in the current game state information, the target game character being game characters controlled by game logic; performing classification on the second feature information to obtain action probability distribution associated with the target game characters; determining target behavior information based on an action probability distribution and target state information associated with the target game character; and controlling the target game character to perform actions corresponding to the target behavior information.

## Claims

### Claim 1

A game character behavior control method, performed by a server, the method comprising:
determining target feature information of a target game character based on parameters of game characters in a game and target state information associated with the target game character in current game state information, the target game character being controlled by a game logic;
performing classification on the target feature information to obtain action probability distribution associated with the target game character;
determining target behavior information based on an action probability distribution and target state information associated with the target game character; and
controlling the target game character to perform actions associated with the target behavior information.

### Claim 2 (depends on 1)

The game character behavior control method according to claim 1, wherein the current game state information of the game is obtained by:
determining a first camp and a second camp in a current game level, the first camp comprising a plurality of game characters, and the second camp comprising a plurality of target game characters;
obtaining category information, attribute information, skill state information, and position information of the game characters in the first camp, and obtaining category information, attribute information, skill state information, and position information of the target game characters in the second camp; and
obtaining the current game state information according to the category information, the attribute information, the skill state information, and the position information of the game characters in the first camp and in the second camp.

### Claim 3 (depends on 1)

The game character behavior control method according to claim 1, further comprising:
performing feature extraction on the current game state information, to obtain first feature information, wherein the first feature information is reflected in the target feature information.

### Claim 4 (depends on 3)

The game character behavior control method according to claim 3, wherein the target feature information is constructed to reflect the first feature information and features of the target state information associated with the target game character in the current game state information.

### Claim 5 (depends on 3)

The game character behavior control method according to claim 3, wherein the first feature information reflects the parameters of the game characters of opposing camps in the game.

### Claim 6 (depends on 3)

The game character behavior control method according to claim 3, wherein performing the feature extraction on the current game state information comprises:
inputting the current game state information into a basic network layer of an actor network model, and performing multi-level feature extraction on the current game state information by using the basic network layer, to obtain the first feature information.

### Claim 7 (depends on 6)

The game character behavior control method according to claim 6, wherein the actor network model further comprises a connection network layer; and the method further comprises constructing second feature information associated with the target game character as the target feature information, comprising:
connecting the first feature information to the category information corresponding to the target game character using the connection network layer, to obtain the second feature information.

### Claim 8 (depends on 7)

The game character behavior control method according to claim 7, wherein the actor network model further comprises a classification network layer; and
the performing classification on the target feature information to obtain an action probability distribution corresponding to the target game character comprises:
inputting the second feature information into the classification network layer, and performing classification on the second feature information by using the classification network layer, to obtain the action probability distribution.

### Claim 9 (depends on 8)

The game character behavior control method according to claim 8, wherein the classification network layer comprises a plurality of sub-classification networks, and a quantity of sub-classification networks is the same as a quantity of target game characters; and
the performing classification on the target feature information by using the classification network layer, to obtain the action probability distributions comprises:
respectively performing classification on the second feature information associated with the target game characters by using the sub-classification networks, to obtain the action probability distributions corresponding to the target game characters.

### Claim 10 (depends on 9)

The game character behavior control method according to claim 9, wherein the determining target behavior information corresponding to the target game character based on the action probability distribution and the target state information comprises:
screening invalid skills in the action probability distribution associated with the target game characters according to skill state information associated with the target game character in the current game state information, to obtain a valid action probability distribution; and
determining the target behavior information corresponding to the target game character according to the corresponding valid action probability distribution.

### Claim 11 (depends on 10)

The game character behavior control method according to claim 10, wherein the determining target behavior information associated with the target game character according to the valid action probability distribution comprises:
obtaining a skill identifier having a maximum probability value in the valid action probability distribution;
dividing the skill identifier having the maximum probability value by a dimension of the skill state information, to obtain a quotient and a remainder; and
determining a target skill according to the remainder, determining a target action character according to the quotient, and determining the target behavior information according to the target skill and the target action character.

### Claim 12 (depends on 1)

The game character behavior control method according to claim 1, further comprising:
forming, a plurality of first lineups and second lineups according to category information of the game characters and a game grouping rule;
arranging the first lineups with the second lineups to form a plurality of first camp groups;
traversing the first camp groups, respectively integrating information of the first camp groups to obtain a first game state information sample; and
performing iterative training on a reinforcement learning network model according to the first game state information sample, to obtain a general-purpose reinforcement learning network model.

### Claim 13 (depends on 12)

The game character behavior control method according to claim 12, further comprising:
identifying a target lineup corresponding to each game level from the second lineups, and arranging the target lineup with each of the first lineups to form a plurality of second camp groups;
traversing the second camp groups, respectively integrating information of the second camp groups to obtain a second game state information sample; and
performing iterative training on the general-purpose reinforcement learning network model according to the second game state information sample, to obtain a specialized reinforcement learning network model.

### Claim 14 (depends on 13)

The game character behavior control method according to claim 13, wherein the reinforcement learning network model comprises an actor network model that is to be trained with a critic network model, the critic network model comprising the basic network layer and a value critic network layer, the value critic network layer being a fully-connected layer with an output dimension of 1.

### Claim 15 (depends on 13)

The game character behavior control method according to claim 13, wherein the performing iterative training on the reinforcement learning network model or the general-purpose reinforcement learning network model according to the first or second game state information sample, respectively, comprises:
iteratively determining a difference between a total reward value and an assessment value based on the first or second game state information sample, and adjusting parameters of the reinforcement learning network model or the general-purpose reinforcement learning network model until the difference is less than a first threshold value or a second threshold value respectively.

### Claim 16 (depends on 15)

The game character behavior control method according to claim 15, wherein iteratively determining a difference between a total reward value and an assessment value comprises, in each iteration:
inputting a first or second game state information sample into the actor network model, performing feature extraction on the first or second game state information by using the actor network model, to obtain behavior information corresponding to the first or the second game state information sample;
obtaining a reward value determined by a game environment according to the behavior information for each turn in a round of battle;
determining, a total reward value corresponding to the first or the second game state information sample according to the reward value for each turn in the round of battle;
inputting the first or the second game state information sample into the critic network model, and performing feature extraction on the first or second game state information sample using the critic network model, to obtain an assessment value corresponding to the first or the second game state information sample; and
determining a difference between the total reward value and the assessment value associated with the first or the second game state information sample, and adjusting parameters of the reinforcement learning network model or the general-purpose reinforcement learning network model respectively.

### Claim 17

A game character behavior control apparatus, comprising: at least one memory and at least one processor coupled to the at least one memory, the at least one processor is configured to:
determine target feature information of a target game character based on parameters of game characters in a game and target state information associated with the target game character in current game state information, the target game character being controlled by a game logic;
perform classification on the second feature information to obtain action probability distribution associated with the target game character;
determine target behavior information based on an action probability distribution and target state information associated with the target game character; and
control the target game character to perform actions associated with the target behavior information.

### Claim 18 (depends on 17)

The game character behavior control apparatus according to claim 17, wherein the at least one processor is configured to:
determine a first camp and a second camp in a current game level, the first camp comprising a plurality of game characters, and the second camp comprising a plurality of target game characters;
obtain category information, attribute information, skill state information, and position information of the game characters in the first camp, and obtaining category information, attribute information, skill state information, and position information of the target game characters in the second camp; and
obtain the current game state information according to the category information, the attribute information, the skill state information, and the position information of the game characters in the first camp and in the second camp.

### Claim 19 (depends on 17)

The game character behavior control apparatus according to claim 17, wherein the at least one processor is configured to:
perform feature extraction on the current game state information, to obtain first feature information, wherein the first feature information is reflected in the target feature information; and
construct the target feature information to reflect the first feature information and features of the target state information associated with the target game character in the current game state information.

### Claim 20

A non-transitory computer-readable storage medium, storing a computer program, the computer program, when executed by at least one processor, implementing:
determining target feature information of a target game character based on parameters of game characters in a game and target state information associated with the target game character in current game state information, the target game character being controlled by a game logic;
performing classification on the target feature information to obtain action probability distribution associated with the target game character;
determining target behavior information based on an action probability distribution and target state information associated with the target game character; and
controlling the target game character to perform actions associated with the target behavior information.