Device and method to localize and control a tool tip with a robot arm
AssigneeHRL Laboratories, LLC
InventorsZhichao Chen, Heiko Hoffmann
Life of this patent
Estimating the term…
Abstract
Described is a system for localizing and controlling a tool tip with a robotic arm. The system receives three-dimensional (3D) visual information of a scene having a tool with a tool tip proximate a target. A depth disparity map is generated based on the 3D visual information. The depth disparity map is filtered by depth continuity to generate boundaries of the tool. The boundaries of the tool are initially loose to ensure that the tool is contained within the boundaries. Thereafter, using level set segmentation, the boundaries of the tool are shrunk and extracted. Ends of the tool tip are identified based on the extracted boundaries, which are then used to generate a pose of the tool tip.
Claims18
A system for localizing and controlling a tool tip with a robotic arm, the system comprising:
- one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
- receiving three-dimensional (3D) visual information of a scene having a tool with a tool tip proximate a target;
- generating a depth disparity map based on the 3D visual information;
- filtering the depth disparity map by depth continuity to generate loose boundaries of the tool;
- shrinking and extracting the boundaries of the tool as an extracted tool boundary;
- identifying ends of the tool tip based on the extracted tool boundary;
- and
- generating a pose of the tool tip.
A computer implemented method for localizing and controlling a tool tip with a robotic arm, the method comprising an act of:
- causing one or more processors to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
- receiving three-dimensional (3D) visual information of a scene having a tool with a tool tip proximate a target;
- generating a depth disparity map based on the 3D visual information;
- filtering the depth disparity map by depth continuity to generate loose boundaries of the tool;
- shrinking and extracting the boundaries of the tool as an extracted tool boundary;
- identifying ends of the tool tip based on the extracted tool boundary;
- and
- generating a pose of the tool tip.
A computer program product for localizing and controlling a tool tip with a robotic arm, the computer program product comprising computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors, such that upon execution of the instructions, the one or more processors perform operations of:
- receiving three-dimensional (3D) visual information of a scene having a tool with a tool tip proximate a target;
- generating a depth disparity map based on the 3D visual information;
- filtering the depth disparity map by depth continuity to generate loose boundaries of the tool;
- shrinking and extracting the boundaries of the tool as an extracted tool boundary;
- identifying ends of the tool tip based on the extracted tool boundary;
- and
- generating a pose of the tool tip.
Dependencies are inferred from the claim text.