Artificial intelligence system for efficiently learning robotic control policies
AssigneeAmazon Technologies, Inc.
InventorsMichael Vogelsong, Darren Ernest Canavor
Life of this patent
Estimating the term…
Abstract
A machine learning system builds and uses control policies for controlling robotic performance of a task. Such control policies may be trained using targeted updates, for example by comparing two trials to identify which represents a greater degree of task success, using this to generate updates from a reinforcement learning system, and weighting the updates based on differences between action vectors of the trials.
Claims21
A method comprising, as implemented by a computing system comprising one or more processors:
- running trials of a robotic system performing a task according to a robotic controller programmed to implement a control policy, wherein the task involves interaction between the robotic system and one or more physical objects;
- identifying one trial of the trials for a pairwise comparison with one other trial of the trials;
- selecting the other trial based at least partly on determining that the other trial is similar to the one trial;
- accessing results of the pairwise comparison to determine which of the one trial and the other trial represents a greater degree of success at the task;
- identifying differences between the one trial and the other trial;
- and
- using machine learning to learn the control policy at least partly by performing targeted updates to the control policy based on the results of the pairwise comparison and the differences between the one trial and the other trial.
A system comprising:
- a robotic system that performs a task according to a robotic controller programmed to implement a control policy;
- and
- a computing system comprising one or more processors, the computing system programmed with executable instructions to at least:
- run trials of the robotic system performing the task using a preliminary version of the control policy;
- identify one trial of the trials for a pairwise comparison with one other trial of the trials;
- select the other trial using k-nearest neighbors to determine that the other trial is similar to the one trial;
- access results of the pairwise comparison to determine which of the one trial and the other trial represents a greater degree of success at the task;
- identify how actions of the robotic system differed between the one trial and the other trial;
- use machine learning to learn the control policy at least partly by performing targeted updates to the control policy based on the results of the pairwise comparison and how the actions differed;
- and
- output the learned control policy to the robotic controller to control the robotic system to perform the task, wherein controlling the robotic system to perform the task comprises controlling physical movement of the robotic system to interact with one or more physical objects.
A non-transitory computer-readable medium storing instructions that, when executed by a computing system comprising one or more processors, cause the computing system to perform operations comprising:
- running trials of a robotic system performing a task according to a control policy, wherein the task involves movement of the robotic system to interact with one or more objects;
- identifying one trial of the trials for a pairwise comparison with one other trial of the trials;
- selecting the other trial based at least partly on determining that the other trial is similar to the one trial;
- accessing results of the pairwise comparison to determine which of the one trial and the other trial represents a greater degree of success at the task;
- identifying differences between the one trial and the other trial;
- and
- performing targeted updates to the control policy based on the results of the pairwise comparison and the differences between the one trial and the other trial.
Dependencies are inferred from the claim text.