US 12,001,320 B2 · Cloud architecture for reinforcement learning

Abstract

The technology described herein provides a cloud reinforcement-learning architecture that allows a single reinforcement-learning model to interact with multiple live software environments. The live software environments and the single reinforcement-learning model run in a distributed computing environment (e.g., cloud environment). The single reinforcement-learning model may run on a first computing device(s) with a graphical processing unit (GPU) to aid in training the single reinforcement-learning model. At a high level, the single reinforcement-learning model may receive state telemetry data from the multiple live environments. The single reinforcement-learning model selects an available action for each set of state telemetry data received and communicates the selection to appropriate the test agent. The test agent then facilitates completion of the action within the software instance being tested in the live environment. A reward is then determined for the action. The single reinforcement-learning model may then be trained.

Claims20

Dependent claims

A method of automated software testing comprising:

selecting, by a reinforcement-learning model, a first action to be taken in a first user interface of a first software instance running in a first live environment;
causing the first action to be performed, wherein the first action includes interacting with a first interface element in the first user interface;
determine a first reward associated with the first action;
selecting, by the reinforcement-learning model, a second action to be taken in the first user interface of a second software instance running in a second live environment;
causing the second action to be performed, wherein the second action includes interacting with a second interface element in the first user interface;
determine a second reward associated with the second action;
generating an updated reinforcement-learning model by training the reinforcement-learning model using the first action, the second action, the first reward, and the second reward;
and
storing the updated reinforcement-learning model.

6 more elements

A computer storage medium storing computer-useable instructions that, when used by a computing device, cause the computing device to perform operations comprising:

receiving, by a reinforcement-learning model, a first image of a first user interface generated by a first instance of the software running in a first live environment;
selecting, by the reinforcement-learning model, a first action to be taken in the first user interface;
causing the first action to be implemented in the first instance of the software, wherein the first action includes interacting with a first interface element in the first user interface to transition the first user interface to a second user interface;
determining, using a second image of the second user interface, a first reward for the first action based on whether the first action achieved one of two or more targeted tasks from a first group of user interface tasks;
receiving, by the reinforcement-learning model, a third image of a third user interface generated by a second instance of the software running in a second live environment;
selecting, by the reinforcement-learning model, a second action to be taken in the third user interface;
causing the second action to be implemented in the second instance of the software, wherein the second action includes interacting with a second interface element in the third user interface to transition the second instance to a fourth user interface;
determining, using a fourth image of the fourth user interface, a second reward for the second action based on whether the second action achieved the one of two or more targeted tasks from the first group of user interface tasks;
generating an updated reinforcement-learning model by training the reinforcement-learning model using the first action, the second action, the first reward, and the second reward;
and
storing the updated reinforcement-learning model.

8 more elements

Dependencies are inferred from the claim text.

Cloud architecture for reinforcement learning

Life of this patent

Cloud architecture for reinforcement learning

Life of this patent

Abstract

Claims20