US 5,631,973 AGranted

Method for telemanipulation with telepresence

AssigneeSRI International

InventorPhilip S. Green

Life of this patent

Estimating the term…

Loading patent…

Abstract

In a telemanipulation system for manipulating objects located in a workspace at a remote worksite by an operator from an operator's station, such as in a remote surgical system, the remote worksite having a manipulator with an end effector for manipulating an object at the workspace, such as a body cavity, a controller including a hand control at the control operator's station for remote control of the manipulator, an image capture device, such as a camera, and image output device for reproducing a viewable real-time image, the improvement wherein a position sensor associated with the image capture device senses position relative to the end effector and a processor transforms the viewable real-time image into a perspective image with correlated manipulation of the end effector by the hand controller such that the operator can manipulate the end effector and the manipulator as if viewing the workspace in true presence. Image transformation according to the invention includes translation, rotation and perspective correction.

Claims12

Dependent claims

In a telemanipulation system for manipulating objects located in a workspace at a remote worksite by a control operator from a control operator station, said telemanipulation system including a manipulator at the worksite with an end effector for manipulation of an object in the workspace, a controller for remote control by the control operator of the manipulator, an image capture device for taking real-time image of the workspace, and an image generator for producing a display for eyes of the control operator, wherein the image capture device is at an angle and a position relative to the end effector which differs from a relative position of eyes of the control operator and the controller, the system further comprising:

an image processing device coupled to the image generator and the image capture device, the image processing device processing the real-time image into a processed image for display to the control operator, wherein the processed image is substantially the same as an image taken by an image capture device located at an angle and position relative to the end effector which is substantially equivalent to a relative angle and position of eyes of the control operator and the controller;
a servo control for controlling position of said manipulator;
a first position sensor disposed to sense a position of said manipulator in said workspace;
a second position sensor disposed to sense a position of said hand control at the operator station;
and
a coordinate transformation device coupled to said image processing device, to said first position sensor and to said second position sensor for transforming information about manipulator position, hand control position and processed image position and into control signals applied to said servo control.

3 more elements

In a telemanipulation system for manipulating objects located in a workspace at a remote worksite by a control operator from a control operator station, said telemanipulation system including a manipulator means at the remote worksite with end effector means for manipulation of an object in the workspace, controller means including hand control means at the control operator station for remote control by the control operator of the manipulator means, image capture means for taking a real-time image of the workspace, and image producing means for producing a display on a monitor for eyes of the control operator, control means comprising:

means coupled to said image capture means for processing the real-time into a processed image for display to the control operator wherein said image capture means is at an angle and a position relative to the end effector means which differs from a relative position of eyes of the control operator and the hand control means, and which further differs in scale and in lateral position, said image processing means correlating manipulation of said end effector means with said hand control means, said image processing means further comprising:
means for performing a perspective correction in the viewable real-time image following a lateral shift and a scale change of the image capture means, wherein said perspective correction means comprises means for performing transformations according to the following relationships:
.alpha.'=arctan [(x'-m)/(y'-n)],
L'=[(x'-m).sup.2 +(y'-n).sup.2 ].sup.1/2,
x'=M(x-.DELTA.x),
y'=M(y-.DELTA.y),
x=L(sin .alpha.)+m,
y=L(cos .alpha.)+n
where
coordinates (m,n) describe a position of a pivot point of the manipulator means before said lateral shift and said scale change occur,
coordinates (m',n') describe a position of said pivot point of the manipulator means after said lateral shift and said scale change occur,
coordinates (x,y) describe a position of the end effector means before said lateral shift and said scale change occur,
coordinates (x',y') describe a position of the end effector means after said lateral shift and said scale change occur,
coordinates (.DELTA.x,.DELTA.y) describe the change in position of the end effector means brought about by said lateral shift,
factor M describes the magnification factor of said scale change,
angle .alpha. is the rotation of the manipulator means about a vertical axis before said lateral shift and said scale change occur,
angle .alpha.' is the rotation of the manipulator means about said vertical axis after said lateral shift and said scale change occur,
length L is a distance between said pivot point and the end effector means before said lateral shift and said scale change occur, and
length L' is a distance between said pivot point and the end effector means after said lateral shift and said scale change occur.

16 more elements

Dependencies are inferred from the claim text.