In-hand pose refinement for pick and place automation
AssigneeIntrinsic Innovation LLC
InventorsRishav Agarwal, Achuta Kadambi, Agastya Kalra, Kartik Venkataraman
Life of this patent
Estimating the term…
Abstract
A method of tracking a pose of an object includes determining an initial pose of the object at a first position, receiving position data and velocity data corresponding to movement of the object to a second position by a moving device, determining an expected pose of the object at the second position based on the position and velocity data and the initial pose, receiving second image data corresponding to the object at the second position from a camera, and determining a refined pose of the object at the second position based on the second image data and the expected pose.
Claims20
A method of tracking a pose of an object, the method comprising:
- moving, by a robotic arm in a workcell, a physical object along a path;
- determining, from one or more first images captured by one or more first cameras in the workcell, an initial pose of the object at a first position on the path;
- receiving position data and velocity data corresponding to movement of the object along the path to a second position on the path by the robotic arm;
- determining an expected pose of the object at the second position on the path based on the position and velocity data and the initial pose of the object;
- receiving second image data captured by one or more second cameras in the workcell, the second image data being image data of the object at the second position;
- and
- determining a refined pose of the object at the second position based on the second image data and the expected pose, including:
- determining expected keypoints on the expected pose of the object and estimated keypoints on the object from the second image data,
- providing the expected pose as an initial condition to a keypoint alignment process to compute an alignment of the expected keypoints and the estimated keypoints, and
- computing the refined pose from the alignment of the expected keypoints and the estimated keypoints.
A pose tracking system comprising:
- a robotic arm;
- and
- one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
- providing instructions to the robotic arm that cause the robotic arm to move a physical object along a path in a workcell;
- determining, from one or more first images captured by one or more first cameras in the workcell, an initial pose of the object at a first position on the path;
- receiving position data and velocity data corresponding to movement of the object along the path to a second position on the path by the robotic arm;
- determining an expected pose of the object at the second position on the path based on the position and velocity data and the initial pose of the object;
- receiving second image data captured by one or more second cameras in the workcell, the second image data being image data of the object at the second position;
- and
- determining a refined pose of the object at the second position based on the second image data and the expected pose, including:
- determining expected keypoints on the expected pose of the object and estimated keypoints on the object from the second image data,
- providing the expected pose as an initial condition to a keypoint alignment process to compute an alignment of the expected keypoints and the estimated keypoints, and
- computing the refined pose from the alignment of the expected keypoints and the estimated keypoints.
Dependencies are inferred from the claim text.