Back to results
US 11,893,468 B2Granted
Imitation learning system
AssigneeNVIDIA Corporation
InventorsYu-Wei Chao, De-An Huang, Christopher Jason Paxton, Animesh Garg, Dieter Fox
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques to identify a goal of a demonstration. In at least one embodiment, video data of a demonstration is analyzed to identify a goal. Object trajectories identified in the video data are analyzed with respect to a task predicate satisfied by a respective object trajectory, and with respect to motion predicate. Analysis of the trajectory with respect to the motion predicate is used to assess intentionality of a trajectory with respect to the goal.
Claims28
A computer-implemented method, comprising:
- segmenting video data into at least a first segment and a second segment the first segment comprising video data representative of a first trajectory of a first object manipulated in a demonstration performed in a first set of circumstances, the second segment comprising video data representative of a second trajectory of a second object manipulated in the demonstration;
- identifying a motion predicate satisfied by the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that movement of the first object on the first trajectory enabled movement of the second object on the second trajectory;
- identifying a task predicate satisfied by the second trajectory, based at least in part on the second trajectory satisfying a logical condition defined in a domain definition;
- identifying a goal of the demonstration based at least in part on the task predicate;
- and
- causing one or more robotic manipulation devices to move from a first pose to a second pose based, at least in part, on performing the goal in a different second set of circumstances.
3 more elements
A system, comprising:
- at least one processor;
- and
- a memory comprising instructions that, as a result of execution by the at least one processor, cause the system to at least:
- segment video data into a first segment and a second segment, the first segment comprising video data representative of a first trajectory of a first object manipulated in a demonstration performed in a first set of circumstances, the second segment comprising video data representative of a second trajectory of a second object manipulated in the demonstration;
- identify a motion predicate satisfied by manipulation of the first object in accordance with the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that manipulation of the first object in accordance with the first trajectory enabled motion of the second object in accordance with the second trajectory;
- identify a task predicate satisfied by manipulation of the second object in accordance with the second trajectory;
- identify a goal of the demonstration based at least in part on the task predicate;
- and
- cause one or more robotic manipulation devices to move from a first pose to a second pose based, at least in part, on performing the goal in a different second set of circumstances.
6 more elements
A non-transitory computer-readable medium comprising instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to:
- divide video data into a first segment and a second segment, the first segment comprising video data representative of a first trajectory of a first object manipulated in a demonstration performed in a first set of circumstances, the second segment comprising video data representative of a second trajectory of a second object;
- identify a motion predicate satisfied by the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that manipulation of the first object enabled the second trajectory;
- identify a task predicate satisfied by the second trajectory;
- identify a goal of the demonstration based at least in part on the task predicate;
- and
- cause one or more robotic manipulation devices to move from a first pose to a second pose based, at least in part, on performing the goal in a different second set of circumstances.
3 more elements
A robotic device, comprising:
- one or more robotic manipulation devices;
- at least one processor;
- and
- a memory comprising instructions that, as a result of execution by the at least one processor, cause the robotic device to at least:
- move the one or more robotic manipulation devices from a first pose to a second pose based, at least in part, on performing a goal in a subsequent set of circumstances based at least in part on a demonstration performed in a previous set of circumstances, wherein the goal is determined by:
- segmenting video data of the demonstration into a first segment and a second segment, the first segment comprising video data representative of a first trajectory of a first object, the second segment comprising video data representative of a second trajectory of a second object;
- identifying a motion predicate satisfied by the first trajectory, wherein the motion predicate is identified based, at least in part, on a determination that the first trajectory enabled the second trajectory;
- identifying a task predicate satisfied by manipulation of the second object in accordance with the second trajectory;
- and
- identifying the goal based at least in part on the task predicate.
7 more elements
Dependencies are inferred from the claim text.