Artificial intelligence system for modeling and evaluating robotic success at task performance
AssigneeAmazon Technologies, Inc.
InventorsBrandon William Porter, Leonardo Ruggiero Bachega, Brian C. Beckman, Benjamin Lev Snyder, Michael Vogelsong, Corrinne Yu
Life of this patent
Estimating the term…
Abstract
A machine learning system builds and uses computer models for identifying how to evaluate the level of success reflected in a recorded observation of a task. Such computer models may be used to generate a policy for controlling a robotic system performing the task. The computer models can also be used to evaluate robotic task performance and provide feedback for recalibrating the robotic control policy.
Claims25
A system comprising:
- a robotic system configured to perform a task according to a policy comprising a set of control parameters;
- an observation system positioned and configured to capture at least video data representing the robotic system performing the task;
- and
- a robotic control system comprising one or more computing devices, the robotic control system programmed with executable instructions to, in real time as the robotic system repeatedly performs the task, monitor the robotic system by at least:
- capturing, using the observation system, a video of the robotic system executing a performance the task;
- evaluating the video using a convolutional neural network trained to determine a level of success of the performance depicted in the video, said convolutional neural network trained with training data that comprises a plurality of pairs of videos of robotic performance of the task, each of said pairs labeled with an indication of which video of the respective pair depicts a higher degree of success;
- comparing the level of success of the performance to a threshold value based on a result of evaluating the video using the convolutional neural network;
- when a result of the comparing indicates that the level of success is above the threshold value, upon the robotic system initiating a further performance of the task, controlling the robotic system using the policy to complete the further performance of the task;
- and
- when a result of the comparing indicates the level of success is below the threshold value:
- recalibrating the policy comprising the set of control parameters using a reinforcement learning system;
- and
- upon the robotic system initiating the further performance of the task, controlling the robotic system using the recalibrated policy to complete the further performance of the task.
A computer-implemented method for real time monitoring of an agent for performance changes due to changing environmental conditions or wear on the agent as the agent repeatedly performs a task, the method comprising, as performed by at least one computing device configured to execute specific instructions:
- accessing a machine learning classifier trained to evaluate whether a recorded observation representing performance of the task depicts success at performing the task, said machine learning classifier trained with training data that comprises a plurality of pairs of videos of agent performance of the task, each of said pairs labeled with an indication of which video of the respective pair depicts a higher degree of success;
- accessing a recorded observation of the agent performing the task based on a policy comprising a set of control parameters;
- evaluating the recorded observation using the machine learning classifier;
- making a determination regarding whether to recalibrate the policy based on a result of evaluating the recorded observation;
- when the determination indicates not to recalibrate the policy, controlling the agent using the policy to complete a further performance of the task;
- and
- when the determination indicates to recalibrate the policy:
- recalibrating the set of control parameters of the policy using a reinforcement learning system;
- and
- upon the agent initiating a further performance of the task, controlling the agent using the recalibrated policy to complete the further performance of the task.
A non-transitory computer storage medium storing executable code, wherein the executable code configures a computing system to perform a process for real time monitoring of a robotic system for performance changes due to changing environmental conditions or wear on the robotic system as the robotic system repeatedly performs a task, the process comprising:
- accessing a plurality of videos of a robotic system performing a task, wherein the plurality videos include a plurality of pairs of videos in which each pair is labeled with an indication of which video of the respective pair depicts the robotic system achieving a higher degree of task success;
- training a machine learning classifier to evaluate success at performance of the task by using the plurality of videos, including the labeled pairs of videos, as at least part of a training data set;
- accessing an additional video of the robotic system performing a further iteration of the task based on a policy comprising a set of control parameters;
- evaluating the additional video using the machine learning classifier;
- making a determination regarding whether to recalibrate the policy based on a result of evaluating the video;
- when the determination indicates not to recalibrate the policy, controlling the agent using the policy to complete a further performance of the task;
- and
- when the determination indicates to recalibrate the policy:
- recalibrating the set of control parameters of the policy using a reinforcement learning system;
- and
- upon the robotic system initiating the further performance of the task, controlling the robotic system using the recalibrated policy to complete the further performance of the task.
Dependencies are inferred from the claim text.