# US 2022/0035640 A1 · Trainable agent for traversing user interface > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0035640 A1 - Document ID: US20220035640A1 - Status: Published application - Kind: A1 - Application number: 16/940,854 - Filed: 2020-07-28 - Published: 2022-02-03 - Earliest claimed: 2020-07-28 - Assignees: Electronic Arts Inc. - Applicants: Electronic Arts Inc. - Inventors: Bijan Daei; Paul Robert Ghita; Ivan Doumenc; Jagtar Gill - Web page: https://patentgrep.com/patent/US20220035640A1 ## CPC - G06F 9/451: Execution arrangements for user interfaces - G06F 11/3696: {Methods or tools to render software testable} - A63F 13/46: Computing the game score - A63F 13/533: for prompting the player, e.g. by displaying a game menu - A63F 13/537: using indicators, e.g. showing the condition of a game character on screen - A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F 11/3698: Environments for analysis, debugging or testing of software - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 3/0499: Feedforward networks - G06N 3/08: Learning methods - G06N 3/092: Reinforcement learning - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An example method of traversing a user interface of an interactive video game by a trainable agent includes: identifying a current observable state of an interactive video game; computing, by a neural network processing the current observable state, a plurality of user interface actions and their respective action scores; selecting, based on the action scores, a user interface action of the plurality of user interface actions; applying the selected user interface action to the interactive video game; and iteratively repeating the computing, selecting, and submitting operations until a desired target observable state of the interactive video game is reached. ## Claims ### Claim 1 A method, comprising: identifying a current observable state of an interactive video game; computing, by a neural network processing the current observable state, a plurality of user interface actions and their respective action scores; selecting, based on the action scores, a user interface action of the plurality of user interface actions; applying the selected user interface action to the interactive video game; and iteratively repeating the computing, selecting, and applying operations until a desired target observable state of the interactive video game is reached. ### Claim 2 (depends on 1) The method of claim 1, wherein selecting the user interface action further comprises: selecting a user interface action that is associated with an optimal action score among the action scores. ### Claim 3 (depends on 1) The method of claim 1, wherein the current observable state of the interactive video game is represented by a numeric vector characterizing one or more parameters of a current graphical user interface (GUI) screen. ### Claim 4 (depends on 1) The method of claim 1, wherein the current observable state of the interactive video game is associated with a reward value, and wherein the neural network is trained to maximize overall reward accumulated by traversing a user interface path to the desired target observable state of the interactive video game. ### Claim 5 (depends on 1) The method of claim 1, further comprising: identifying the neural network among a plurality of neural networks associated with the interactive video game, by matching a version identifier of the neural network to a version identifier of the interactive video game. ### Claim 6 (depends on 1) The method of claim 1, further comprising: responsive to detecting an error in the interactive video game, modifying one or more parameters of the neural network. ### Claim 7 (depends on 1) The method of claim 1, further comprising: responsive to failing to achieve the desired observable state of the interactive video game within a predefined number of iterations, modifying one or more parameters of the neural network. ### Claim 8 (depends on 1) The method of claim 1, further comprising: training the neural network by a reinforcement learning process. ### Claim 9 A system, comprising: a memory; and a processor, communicatively coupled to the memory, the processor configured to: identify a current observable state of an interactive video game; compute, by a neural network processing the current observable state, a plurality of user interface actions and their respective action scores; select, based on the action scores, a user interface action of the plurality of user interface actions; apply the selected user interface action to the interactive video game; and iteratively repeat the computing, selecting, and applying operations until a desired target observable state of the interactive video game is reached. ### Claim 10 (depends on 9) The system of claim 9, wherein the interactive video game is an interactive video game. ### Claim 11 (depends on 9) The system of claim 9, wherein selecting the user interface action further comprises: selecting a user interface action that is associated with an optimal action score among the action scores. ### Claim 12 (depends on 9) The system of claim 9, wherein the current observable state of the interactive video game is represented by a numeric vector characterizing one or more parameters of a current graphical user interface (GUI) screen. ### Claim 13 (depends on 9) The system of claim 9, wherein the processor is further configured to: identify the neural network among a plurality of neural networks associated with the interactive video game, by matching a version identifier of the neural network to a version identifier of the interactive video game. ### Claim 14 (depends on 9) The system of claim 9, wherein the processor is further configured to: responsive to detecting an error in the interactive video game, modify one or more parameters of the neural network. ### Claim 15 (depends on 9) The system of claim 9, wherein the processor is further configured to: responsive to failing to achieve the desired observable state of the interactive video game within a predefined number of iterations, modify one or more parameters of the neural network. ### Claim 16 A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a computing device, cause the computing device to: identify a current observable state of an interactive video game; compute, by a neural network processing the current observable state, a plurality of user interface actions and their respective action scores; select, based on the action scores, a user interface action of the plurality of user interface actions; apply the selected user interface action to the interactive video game; and iteratively repeat the computing, selecting, and applying operations until a desired target observable state of the interactive video game is reached. ### Claim 17 (depends on 16) The computer-readable non-transitory storage medium of claim 16, wherein selecting the user interface action further comprises: selecting a user interface action that is associated with an optimal action score among the action scores. ### Claim 18 (depends on 16) The computer-readable non-transitory storage medium of claim 16, wherein the current observable state of the interactive video game is represented by a numeric vector characterizing one or more parameters of a current graphical user interface (GUI) screen. ### Claim 19 (depends on 16) The computer-readable non-transitory storage medium of claim 16, wherein the current observable state of the interactive video game is associated with a reward value, and wherein the neural network is trained to maximize overall reward accumulated by traversing a user interface path to the desired target observable state of the interactive video game. ### Claim 20 (depends on 16) The computer-readable non-transitory storage medium of claim 16, further comprising executable instructions that, when executed by the computing device, cause the computing device to: identify the neural network among a plurality of neural networks associated with the interactive video game, by matching a version identifier of the neural network to a version identifier of the interactive video game.