Systems and methods of interacting with a robotic tool using free-form gestures
AssigneeUltrahaptics IP Two Limited
InventorsMaxwell Sills, Robert S. Gordon, Paul Durdik
Life of this patent
Estimating the term…
Abstract
The technology disclosed relates to motion capture and gesture recognition. In particular, it calculates the exerted force implied by a human hand motion and applies the equivalent through a robotic arm to a target object. In one implementation, this is achieved by tracking the motion and contact of the human hand and generating corresponding robotic commands that replicate the motion and contact of the human hand on a workpiece through a robotic tool.
Claims20
A method of using gestures to control a robotic tool to manipulate a workpiece, the method including:
- capturing sequential images of a hand in a three-dimensional (3D) sensory space while the hand is (i) empty and (ii) making motions directed to commanding the robotic tool;
- recognizing, from the captured images of the hand while empty, a gesture segment of the hand that represents a manipulation of a workpiece by the robotic tool, wherein there is no physical contact between the hand and any object during the recognized gesture segment of the hand;
- determining, according to the recognized gesture segment, a command to cause the robotic tool to apply a force to the workpiece without actual physical contact between the robotic tool and the hand, wherein a magnitude of the force to be applied to the workpiece is determined based on a motion of the hand during the recognized gesture segment;
- and
- issuing the determined command to the robotic tool to apply the force to the workpiece.
A method of using gestures to control a robotic tool to manipulate a workpiece, the method including:
- capturing sequential images of a hand in a three-dimensional (3D) sensory space while the hand is (i) interacting with a manipulable object and (ii) making motions directed to commanding the robotic tool;
- recognizing, from the captured images of the hand, a gesture segment of the hand that represents physical a manipulation of a workpiece by the robotic tool, wherein there is neither physical nor electrical connection facilitating passage of a signal between the manipulable object and the robotic tool during the recognized gesture segment;
- determining, according to the recognized gesture segment, a command to cause the robotic tool to apply a force to the workpiece without actual physical contact between the robotic tool and the hand, wherein a magnitude of the force to be applied to the workpiece is determined based on motion of the hand during the recognized gesture segment;
- and
- issuing the determined command to the robotic tool to apply the force to the workpiece.
A method of a robotic tool manipulating a workpiece, the method including:
- receiving, by the robotic tool, a command to cause the robotic tool to apply a force to the workpiece without actual physical contact between the robotic tool and a hand of a user, the received command being determined by:
- capturing sequential images of a hand in a three-dimensional (3D) sensory space while the hand is (i) empty and (ii) making motions directed to commanding the robotic tool;
- and
- recognizing, from the captured images of the hand while empty, a gesture segment of the hand that represents a manipulation of a workpiece by the robotic tool, wherein there is no physical contact between the hand and any object during the recognized gesture segment;
- and
- applying the force to the workpiece, according to the received command, without actual physical contact between the robotic tool and the hand, wherein a magnitude of the force applied to the workpiece is determined based on a motion of the hand during the recognized gesture segment.
Dependencies are inferred from the claim text.