Back to results
US 12,112,259 B1Granted
Dynamic environment configurations for machine learning services
AssigneeAmazon Technologies, Inc.
InventorLeo Parker Dirac
Life of this patent
Estimating the term…
Abstract
Features related to systems and methods for reinforcement learning are described. The environment includes one or more agents for automating the training of reinforcement learning (RL) models. The environment may include a simulator or real-world observations. The features described identify key training parameters, resource configurations, virtual network configurations, and simulators based on historical training data.
Claims19
A system for generating trained reinforcement learning models, the system comprising:
- a plurality of processing devices corresponding to a simulation environment for a set of simulated, hosted customer networks;
- and
- a non-transitory computer readable medium storing instructions that, when executed by the at least a first processing device, cause the system to perform operations including:
- receiving, from a client device, a request to provide a generated trained reinforcement learning model on training data generated from an identified first simulated, hosted customer network;
- responsive to the request from the client device, selecting a first simulated, hosted customer network from a set of simulated, hosted customer networks using the received request as selection criteria;
- identifying, from a data store, a simulator for generating the training data corresponding to a network simulation of an identified network and for training a reinforcement learning model based at least in part on the request;
- identifying a reference reinforcement learning model for the request;
- instantiating a simulation environment based at least in part on a parameter from an environment used to train the reference reinforcement learning model;
- instantiating an agent in the simulation environment including the simulator, the agent being configured using at least one parameter used to train the reference reinforcement learning model;
- activating the agent for a simulation period to form the trained reinforcement learning model, wherein each activation includes a state and a reward for a previous action taken by the agent;
- and
- providing to a client device the generated trained reinforcement learning model, of a simulated reinforcement learning model of the training data, in response to the request.
9 more elements
A system comprising:
- a plurality of processing devices corresponding to a simulation environment for a set of simulated, hosted customer networks;
- and
- a non-transitory computer readable medium storing instructions that, when executed by the at least a first processing device, cause the system to:
- receive, from a client device, a request to provide a generated trained reinforcement learning model on training data generated from an identified simulated, hosted customer network;
- responsive to the request from the client device, selecting a first simulated, hosted customer network from a set of simulated, hosted customer networks using the received request as selection criteria;
- identify, from a data store, an environment for generating the training data corresponding to a network simulation of an identified network and for training a reinforcement learning model based at least in part on the request;
- identify a reference reinforcement learning model for the training request;
- instantiate a simulation environment interface based at least in part on a parameter from an environment used to train the reference reinforcement learning model, wherein the environment interface communicates with the simulation environment;
- instantiate an agent in communication with the environment interface, the agent being configured using at least one parameter used to train the reference reinforcement learning model;
- activate the agent for a simulation period to form the trained reinforcement learning model, wherein each activation includes a state and a reward for a previous action taken by the agent;
- and
- provide to a client device the generated trained reinforcement learning model, of a simulated reinforcement learning model of the training data, in response to the request.
9 more elements
A computer-implemented method comprising:
- under control of a plurality of processing devices corresponding to a simulation environment for a set of simulated, hosted customer networks,
- identifying, from a data store, an environment for training a reinforcement learning model based at least in part on a training request;
- identifying a reference reinforcement learning model for generating the training data corresponding to a network simulation of an identified network and for the training request on the training data generated from an identified simulated, hosted customer network;
- responsive to the training request, selecting a first simulated, hosted customer network from a set of simulated, hosted customer networks using the training request as selection criteria;
- instantiating an environment interface based at least in part on a parameter from an environment used to train the reference reinforcement learning model, wherein the environment interface communicates with the environment;
- instantiating an agent in communication with the environment interface to generate a trained reinforcement learning model, the agent being configured using at least one parameter used to train the reference reinforcement learning model;
- activating the agent for a simulation period, wherein each activation includes a state and a reward for a previous action taken by the agent;
- and
- providing to a client device the generated trained reinforcement learning model, of a simulated reinforcement learning model of the training data, in response to the training request.
6 more elements
Dependencies are inferred from the claim text.