Back to results
US 10,766,136 B1Granted
Artificial intelligence system for modeling and evaluating robotic success at task performance
AssigneeAmazon Technologies, Inc.
InventorsBrandon William Porter, Leonardo Ruggiero Bachega, Brian C. Beckman, Benjamin Lev Snyder, Michael Vogelsong, Corrinne Yu
Life of this patent
Estimating the term…
Abstract
A machine learning system builds and uses computer models for identifying how to evaluate the level of success reflected in a recorded observation of a task. Such computer models may be used to generate a policy for controlling a robotic system performing the task. The computer models can also be used to evaluate robotic task performance and provide feedback for recalibrating the robotic control policy.
Claims15
A system comprising:
- a robotic system configured to perform a task according to a set of control parameters, the task comprising picking up and moving a particular type of physical object;
- an observation system positioned and configured to capture at least video data representing the robotic system performing the task, the video data comprising frames showing movement of a physical object of the particular type by the robotic system;
- and
- a robotic control system comprising one or more computing devices, the robotic control system programmed with executable instructions to at least:
- access a corpus of training videos depicting performances of the task;
- train a convolutional neural network to evaluate a level of success depicted in a given video of the task, wherein the convolutional neural network, during training, learns what pixel values, sequences of pixel values, or vector values in vector representations of the training videos, correspond to success at task performance, and generates weights that encode information about the pixel values, sequences of pixel values, or vector values corresponding to success at task performance;
- capture, using the observation system, a plurality of videos of the robotic system performing the task, each such video comprising frames showing movement by the robotic system of a physical object of the particular type;
- evaluate the plurality of videos using the convolutional neural network, wherein evaluating the plurality videos comprises using the weights of the convolutional neural network to generate data values representing preferences for particular videos over other videos, said data values specifying, for each of a plurality of pairs of the videos compared with the convolutional neural network, which video of the respective pair represents a higher level of success at performing the task;
- learn a reward function modeling success at performance of the task based at least partly on the data values generated by the convolutional neural network, wherein the reward function is configured to generate a reward value representing a level of success of task performance depicted in a video;
- generate, using a reinforcement learning system and the reward function, a policy comprising the set of control parameters;
- and
- upon the robotic system initiating a further performance of the task, control the robotic system using the policy to complete the further performance of the task.
9 more elements
A computer-implemented method comprising, as performed by at least one computing device configured to execute specific instructions:
- accessing a machine learning classifier trained to evaluate whether a recorded observation representing performance of a task depicts success at performing the task, the task comprising movement by a robotic system of a physical object of a particular type, the machine learning classifier comprising weights that encode information about what pixels, pixel values, or vector values in vector representations of videos, correspond to success at performance of the task;
- accessing first and second recorded observations of robotic system performance of the task, the recorded observations comprising video frames showing movement of a physical object of the particular type;
- comparing the first recorded observation to the second recorded observation using the machine learning classifier, wherein comparing the first and second recorded observations comprises applying the weights of the machine learning classifier to the first and second recorded observations and outputting an indication of which of said recorded observations depicts a higher level of success at performing the task;
- determining a reward value based at least partly on a result of the comparison, the reward value representing a level of success at performance of the task;
- generating, using a reinforcement learning system and the reward value, a policy for controlling the robotic system;
- and
- controlling operations of the robotic system performing the task using the policy.
4 more elements
A non-transitory computer storage medium storing executable code, wherein the executable code configures a computing system to perform a process comprising:
- accessing a corpus of training data depicting performances of a task in which a robotic system moves a physical object;
- training a machine learning classifier to evaluate success at performance of the task represented in the training data, wherein training the machine learning classifier comprises generating weights that encode information about pixel values, sequences of pixel values, or vector values in vector representations of videos, that correspond to success at performance of the task;
- by the trained machine learning classifier, evaluating recorded observations of robotic performances of the task, wherein evaluating the recorded observations comprises using the weights to generate data values indicating, for each of a plurality of pairs of the recorded observations compared with the trained machine learning classifier, which recorded observation of the respective pair depicts a higher level of success at performing the task;
- building, based at least partly on the data values generated by the trained machine learning classifier, a reward function to model parameters influencing success at performance of the task, the reward function configured to generate for a recorded observation a reward value representing a level of success at performance of the task;
- and
- providing the reward function to a reinforcement learning system configured to optimize a control policy for a robotic system designated to perform the task.
3 more elements
Dependencies are inferred from the claim text.