Motion guidance and natural language commands based robotic systems
AssigneesToyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
InventorThomas Kollar
Life of this patent
Estimating the term…
Abstract
A robotic system is contemplated. The robotic system comprises a robot comprising a camera, a microphone, memory, and a controller that is configured to receive a natural language command for performing an action within a real world environment, parse the natural language command, categorize the action as being associated with guidance for performing the action, receive the guidance for performing the action, the guidance including a motion applied to at least one portion of the robot within the real world environment for performing the action, and store, in the memory, the natural language command in correlation with the motion that is applied to the at least one portion of the robot.
Claims18
A method implemented by a computing device of a robot, the method comprising:
- receiving a natural language command for performing an action within a real world environment, the natural language command including one or more terms;
- parsing, by the computing device, the natural language command by analyzing the one or more terms of the natural language command to identify the action;
- categorizing the action as being associated with guidance for performing the action;
- receiving the guidance for performing the action, the guidance including a motion applied to at least one portion of the robot within the real world environment for performing the action;
- storing the natural language command in correlation with the motion in memory of the computing device;
- receiving a future natural language command at a time point subsequent to the natural language command and accessing the memory to determine a degree of similarity relative to the stored natural language command in correlation with the motion to perform the action associated with the future natural language command;
- outputting a voice command indicating a request for clarification of the natural language command, wherein the voice command is outputted in response to determining an absence of a record of the action in the memory of the computing device to be performed in correlation with the motion;
- entering, by the robot and in response to putting the voice command indication the request for clarification of the natural language command, a training mode including storing the motion applied to the at least one portion of the robot;
- and
- performing, in response to the robot entering the training mode, the action specified in the natural language command stored in the memory of the computing device automatically and without user intervention.
A robotic system comprising:
- a robot comprising:
- a camera;
- a microphone;
- memory;
- and
- a controller that is configured to:
- receive a natural language command for performing an action within a real world environment, the natural language command including one or more terms,
- parse the natural language command by analyzing the one or more terms of the natural language command to identify the action,
- categorize the action as being associated with guidance for performing the action,
- receive the guidance for performing the action, the guidance including a motion applied to at least one portion of the robot within the real world environment for performing the action,
- store the natural language command in correlation with the motion in the memory,
- receive a future natural language command at a time point subsequent to the natural language command and access the memory to determine a degree of similarity relative to the stored natural language command in correlation with the motion to perform the action associated with the future natural language command,
- output a voice command indicating a request for clarification of the natural language command, wherein the voice command is outputted in response to determining an absence of a record of the action in the memory to be performed in correlation with the motion;
- enter, by the robot and in response to putting the voice command indication the request for clarification of the natural language command, a training mode including storing the motion applied to the at least one portion of the robot;
- and
- perform, in response to the robot entering the training mode, the action specified in the natural language command stored in the memory automatically and without user intervention.
Dependencies are inferred from the claim text.