Minimally invasive telesurgical systems with interactive user interfaces for 3D operative images

A minimally invasive surgical system comprising:

• a stereoscopic endoscope to capture stereo video images of a surgical site and one or more minimally invasive surgical instruments;
• a stereoscopic video display device of a surgeon console to display stereo video images to a user to view images with a three dimensional (3D) appearance;
• a processor coupled to the stereoscopic endoscope and the stereoscopic video display device, in response to stored program instructions, the processor configured to
• generate an operative image of an anatomic structure in the surgical site,
• overlay the operative image onto the captured stereo video images for display on the stereoscopic video display device,
• generate and overlay a three dimensional pointer onto the operative image or the captured stereo video images to display the three dimensional pointer on the stereoscopic video display device, wherein the three dimensional pointer is generated and overlaid onto the operative image or the captured stereo video images to display the three dimensional pointer on the stereoscopic video display device with a three dimensional appearance, the three dimensional pointer adapted to change its size relative to its depth in the operative image, and
• switch from a first mode for one or more input devices of a surgeon console into a second mode for the one or more input devices, wherein the first mode is used to couple motion into minimally invasive surgical instruments, wherein the second mode is used to control an interactive graphical user interface including a tool icon, overlaid onto the images of the surgical site, with context sensitive menu items, the context sensitive menu items changing depending upon the surgical instrument or function selected, to allow interactions with the three dimensional pointer and the operative image in three dimensions using the one or more input devices.