Back to results
US 11,900,244 B1Granted
Attention-based deep reinforcement learning for autonomous agents
AssigneeAmazon Technologies, Inc.
InventorsSahika Genc, Sravan Babu Bodapati, Tao Sun, Sunil Mallya Kasaragod
Life of this patent
Estimating the term…
Abstract
A data source configured to provide a representation of an environment of one or more agents is identified. Using a data set obtained from the data source, a neural network-based reinforcement learning model with one or more attention layers is trained. Importance indicators generated by the attention layers are used to identify actions to be initiated by an agent. A trained version of the model is stored.
Claims20
A system, comprising:
- one or more computing devices;
- wherein the one or more computing devices include instructions that upon execution on or across one or more processors cause the one or more computing devices to:
- identify, based at least in part on a first programmatic interaction, a first simulator of an environment of one or more autonomous agents;
- train, using at least a pixel-level data set obtained from the first simulator as input, a neural network-based reinforcement learning model, wherein the neural network-based reinforcement learning model includes at least (a) a perception sub-model with one or more convolution layers and one or more attention layers and (b) a dynamics sub-model, wherein the perception sub-model is trained jointly with the dynamics sub-model, wherein the dynamics sub-model identifies one or more actions resulting in a change in vehicle motion to be initiated by an autonomous agent based on output generated by the perception sub-model, wherein the output comprises relative importance indicators assigned to respective elements of the environment, and wherein the relative importance indicators are determined by the one or more attention layers;
- store a trained version of the neural network-based reinforcement learning model;
- and
- transmit, based at least in part on a second programmatic interaction, the trained version of the neural network-based reinforcement learning model to a target autonomous agent, wherein one or more actions of the target autonomous agent within a non-simulation environment are initiated using the trained version of the neural network-based reinforcement learning model, and wherein at least one action of the one or more actions results in a change in motion of a vehicle at least partially controlled by the target autonomous agent.
4 more elements
A method, comprising:
- performing, by one or more computing devices:
- identifying a data source which provides a representation of a non-simulation environment of one or more autonomous agents;
- training, using a data set obtained at least in part from the data source, a neural network-based reinforcement learning model, wherein the neural network-based reinforcement learning model includes a subcomponent comprising one or more attention layers configured to assign relative importance indicators to respective elements of the non-simulation environment, and wherein the relative importance indicators are used at one or more other sub-components of the neural network-based reinforcement learning model to identify one or more actions that cause, at least in part, an autonomous agent to initiate a change in vehicle motion, wherein the subcomponent comprising one or more attention layers is trained jointly with the one or more other sub-components of the neural network-based reinforcement learning model that identifies the one or more actions;
- storing a trained version of the neural network-based reinforcement learning model;
- and
- transmitting the trained version of the neural network-based reinforcement learning model to a target autonomous agent, wherein one or more actions of the target autonomous agent within the non-simulation environment are initiated using the trained version of the neural network-based reinforcement learning model, and wherein at least one action of the one or more actions cause, at least in part, the change in vehicle motion.
3 more elements
One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause one or more computer systems to:
- identify a first data source configured to provide a representation of a non-simulation environment of one or more autonomous agents;
- train, using a data set obtained at least in part from the first data source, a neural network-based reinforcement learning model, wherein the neural network-based reinforcement learning model includes a subcomponent comprising one or more attention layers configured to assign importance indicators to respective elements of the non-simulation environment, wherein one or more of the importance indicators produced by the one or more attention layers are used at one or more other sub-components of the neural network-based reinforcement learning model to identify one or more actions that cause, at least in part, an autonomous agent to initiate a change in vehicle motion, wherein the subcomponent comprising one or more attention layers is trained jointly with the one or more other sub-components of the neural network-based reinforcement learning model that identifies the one or more actions;
- store a trained version of the neural network-based reinforcement learning model and
- transmit the trained version of the neural network-based reinforcement learning model to a target autonomous agent, wherein one or more actions of the target autonomous agent within the non-simulation environment are initiated using the trained version of the neural network-based reinforcement learning model, and wherein at least one action of the one or more actions cause, at least in part, the change in vehicle motion.
1 more element
Dependencies are inferred from the claim text.