# US 2024/0198232 A1 · Efficient gameplay training for artificial intelligence

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0198232 A1
- Document ID: US20240198232A1
- Status: Published application
- Kind: A1
- Application number: 18/554,931
- Filed: 2022-04-11
- Published: 2024-06-20
- Earliest claimed: 2021-05-26
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Nathan Sun Martz; Horacio Hernan Moraldo; Stewart Miles; Leopold Haller; Hinako Sakazaki
- Web page: https://patentgrep.com/patent/US20240198232A1

## CPC

- A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
- A63F 13/352: involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
- A63F 13/79: involving player-related data, e.g. identities, accounts, preferences or play histories
- G06N 5/04: Inference or reasoning models
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods are described for training a locally executed actor component to execute real-time gameplay actions in a gaming application based on one or more gameplay data models generated by a remote learning service. A gameplay data model for the gaming application is provided from one or more server computing systems executing the remote learning service to the client computing device. Observational data is generated by the local actor component based on in-game results of artificial gameplay actions performed by the local actor component, based at least in part on inferences generated by the actor component using the provided gameplay data model. Based on the received observational data, the remote learning service modifies the gameplay data model and provides the modified gameplay data model to the local actor component to improve future artificial gameplay actions.

## Claims

### Claim 1

A server method, comprising:
providing, from one or more server computing systems to a remote client computing device and via a programmatic interface, a gameplay data model for a gaming application executing on the remote client computing device;
receiving, from the remote client computing device via the programmatic interface, observational data generated from artificial gameplay actions performed within the gaming application by an actor component executing on the remote client computing device and based at least in part on inferences generated by the actor component using the provided gameplay data model;
modifying, by the one or more server computing systems, the gameplay data model based on the received observational data; and
providing, to the remote client computing device and via the programmatic interface, the modified gameplay data model.

### Claim 2 (depends on 1)

The method of claim 1, further comprising receiving, by the one or more server computing systems and via the programmatic interface, control information associating each of one or more output states of the gaming application with an input variable of the actor component executing on the remote client computing device.

### Claim 3 (depends on 2)

The method of claim 2, wherein the one or more output states of the gaming application include one or more of a group that includes a player reference position within a virtual environment of the gaming application, a position of an object relative to the player reference position within the virtual environment of the gaming application, a motion vector associated with an object relative to the player reference position within the virtual environment of the gaming application, geometry information regarding one or more aspects of the virtual environment of the gaming application, and/or one or more in-game reward indicators associated with gameplay of the gaming application.

### Claim 4 (depends on 1)

The method of claim 1, further comprising receiving, by the one or more server computing systems, control information associating each of one or more output variables for the actor component with an action available to a human user of the gaming application.

### Claim 5 (depends on 1)

The method of claim 1, wherein modifying the gameplay data model is further based on additional observational data generated based on gameplay actions performed within the gaming application by a human user of the gaming application.

### Claim 6 (depends on 5)

The method of claim 5, wherein modifying the gameplay data model based on the additional observational data includes modifying the gameplay data model using a deep learning artificial intelligence.

### Claim 7 (depends on 1)

The method of claim 1, further comprising generating test data for the gaming application based on the artificial gameplay actions.

### Claim 8 (depends on 1)

The method of claim 1, further comprising modifying the gameplay data model based on the received observational data in response to having received an aggregation of observational data meeting at least one predefined criterion.

### Claim 9 (depends on 8)

The method of claim 8, wherein the at least on criterion comprises at least one of a defined duration period, a defined quantity of observational data and an explicit request received at the remote client computing device.

### Claim 10

(canceled)

### Claim 11

(canceled)

### Claim 12

A server, comprising:
a network interface;
one or more processors; and
a memory storing a set of executable instructions, the set of executable instructions to manipulate the one or more processors to:
generate, based at least in part on control information associating each of one or more output states of a gaming application with an input variable, a gameplay data model for the gaming application;
provide, via a programmatic interface, the generated gameplay data model to an actor component executing on a remote client computing device;
receive, from the actor component and via the programmatic interface, observational data generated from artificial gameplay actions performed within the gaming application by the actor component based on inferences generated by the actor component using the generated gameplay data model;
modify the generated gameplay data model based on the received observational data; and
provide, to the actor component and via the programmatic interface, the modified gameplay data model for use by the actor component in performing additional artificial gameplay actions within the gaming application.

### Claim 13 (depends on 12)

The server of claim 12, wherein the remote client computing device executes an instance of the gaming application, and wherein the observational data is generated from artificial gameplay actions performed by the actor component within the instance of the gaming application executed by the remote client computing device.

### Claim 14 (depends on 12)

The server of claim 12, wherein the set of executable instructions is further to manipulate the one or more processors to receive, via the programmatic interface, control information associating each of one or more output states of the gaming application with an input variable of the actor component executing on the remote client computing device.

### Claim 15 (depends on 14)

The server of claim 14, wherein the one or more output states of the gaming application include one or more of a group that includes a player reference position within a virtual environment of the gaming application, a position of an object relative to the player reference position within the virtual environment of the gaming application, a motion vector associated with an object relative to the player reference position within the virtual environment of the gaming application, geometry information regarding one or more aspects of the virtual environment of the gaming application, and/or one or more in-game reward indicators associated with gameplay of the gaming application.

### Claim 16 (depends on 12)

The server of claim 12, wherein the set of executable instructions is further to manipulate the one or more processors to receive, via the programmatic interface, control information associating each of one or more output variables for the actor component with an action available to a human user of the gaming application.

### Claim 17 (depends on 12)

The server of claim 12, wherein the set of executable instructions is further to manipulate the one or more processors to receive, via the programmatic interface, additional observational data generated from gameplay actions performed within the gaming application by a human user of the gaming application, and wherein to modify the gameplay data model is further based on the received additional observational data.

### Claim 18 (depends on 17)

The server of claim 17, wherein to modify the gameplay data model based on the received additional observational data includes to modify the gameplay data model using a deep learning artificial intelligence.

### Claim 19

A method, comprising:
receiving, by an actor component executed by one or more processors and via a programmatic interface from a machine learning component executing on one or more remote server computing systems, a gameplay data model for a gaming application;
executing, by the one or more processors, an instance of the gaming application;
providing, to the machine learning component and via the programmatic interface, observational data generated from artificial gameplay actions performed within the executing instance of the gaming application by the actor component based at least in part on inferences generated by the actor component using the gameplay data model; and
receiving, from the machine learning component executing on the one or more remote server computing systems and via the programmatic interface, a modified gameplay data model based at least in part on the provided observational data.

### Claim 20 (depends on 19)

The method of claim 19, further comprising performing one or more additional artificial gameplay actions based at least in part on additional inferences generated by the actor component using the modified gameplay data model.

### Claim 21 (depends on 19)

The method of claim 19, further comprising generating test data for the gaming application based on the artificial gameplay actions.

### Claim 22 (depends on 19)

The method of claim 19, wherein the gameplay data model is based at least in part on control information associating each of one or more output states of the gaming application with an input variable of the actor component.

### Claim 23 (depends on 22)

The method of claim 22, wherein the one or more output states of the gaming application include one or more of a group that includes a player reference position within a virtual environment of the gaming application, a position of an object relative to the player reference position within the virtual environment of the gaming application, a motion vector associated with an object relative to the player reference position within the virtual environment of the gaming application, geometry information regarding one or more aspects of the virtual environment of the gaming application, and/or one or more in-game reward indicators associated with gameplay of the gaming application.

### Claim 24 (depends on 19)

The method of claim 19, wherein the gameplay data model is based at least in part on control information associating each of one or more output variables for the actor component with an action available to a human user of the gaming application.

### Claim 25 (depends on 19)

The method of claim 19, further comprising generating additional observational data generated from gameplay actions performed within the gaming application by a human user of the gaming application, such that the modified gameplay data model is further based on the additional observational data.

### Claim 26

(canceled)

### Claim 27

(canceled)
