Machine learning control of object handovers
AssigneeNVIDIA Corporation
InventorsWei Yang, Christopher Jason Paxton, Yu-Wei Chao, Dieter Fox
Life of this patent
Estimating the term…
Abstract
A robotic control system directs a robot to take an object from a human grasp by obtaining an image of a human hand holding an object, estimating the pose of the human hand and the object, and determining a grasp pose for the robot that will not interfere with the human hand. In at least one example, a depth camera is used to obtain a point cloud of the human hand holding the object. The point cloud is provided to a deep network that is trained to generate a grasp pose for a robotic gripper that can take the object from the human's hand without pinching or touching the human's fingers.
Claims26
A processor, comprising one or more computers comprising one or more processors to:
- obtain a point cloud that represents a hand holding an object;
- determine, from a first portion of the point cloud, a pose of the object;
- determine, from a second portion of the point cloud, a pose of the hand;
- generate a set of grasp poses that allow a robot to grasp the object;
- select, from the set of grasp poses, based at least in part on the pose of the hand, a target grasp pose that does not interfere with the hand;
- and
- cause the robot to perform the target grasp pose.
A system, comprising:
- one or more processors coupled to non-transitory computer-readable media;
- the non-transitory computer-readable media storing executable instructions that, as a result of being executed by the one or more processors, cause the system to:
- determine, from a three-dimensional image of an appendage holding an object, a pose of the appendage and a pose of the object;
- determine a set of grasp poses that allow a robotic gripper to grasp the object;
- select, from the set of grasp poses, a grasp pose that does not interfere with the appendage;
- and
- perform the grasp pose.
A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
- obtain a three-dimensional image of an appendage holding an object;
- generate a 3D model from the three dimensional image;
- provide the 3D model to a trained network, the trained network producing a grasp pose for a robotic gripper that is able to grasp the object without contacting the appendage;
- and
- cause the robotic gripper to perform the grasp pose.
A processor comprising:
- one or more arithmetic logic units (ALUs) to train one or more neural networks, at least in part, by providing, to a network, training data that includes a 3D model of a appendage holding an object, and a proposed grasp for the robotic gripper such that the robotic gripper can receive the object from the appendage without interfering with the appendage.
Dependencies are inferred from the claim text.