# US 2019/0167370 A1 · System and method for controlling a remote medical device guidance system in three-dimensions using gestures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0167370 A1 - Document ID: US20190167370A1 - Status: Published application - Kind: A1 - Application number: 16/271,122 - Filed: 2019-02-08 - Published: 2019-06-06 - Earliest claimed: 2009-07-22 - Assignees: St. Jude Medical, Atrial Fibrillation Division, Inc. - Applicants: St. Jude Medical, Atrial Fibrillation Division, Inc. - Inventors: Eric S. Olson - Web page: https://patentgrep.com/patent/US20190167370A1 ## CPC - A61B 34/74: {Manipulators with manual electric input means} - A61B 34/25: {User interfaces for surgical systems} - A61B 34/76: {Manipulators having means for providing feel, e.g. force or tactile feedback} - B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor} - G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/03545: {Pens or stylus} - G06T 19/003: {Navigation within 3D models or images} - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - A61B 5/283: Invasive - A61B 34/30: Surgical robots - A61B 2034/258: {providing specific settings for specific users} - A61B 2034/741: {Glove like input devices, e.g. "data gloves"} - G06T 2210/41: Medical - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A system for enabling a user to remotely control a robotic medical device system includes a motion capture apparatus to capture motion of a user in a sensing volume and generate indicative output data. The system includes a control unit configured to execute gesture recognition logic that recognizes a user gesture based on analysis of the indicative output data. The control unit executes interpreter logic that is configured to translate the recognized user gesture into a corresponding robotic medical device control command configured to control an aspect of the operation of the robotic medical device system. ## Claims ### Claim at position 1 1-20. (canceled) ### Claim 21 A medical apparatus comprising: an electronic control unit and a memory; a robotic medical device system in communication with said electronic control unit; and a bedside interface device comprising a touch-responsive display panel configured to allow a user to communicate with a plurality of diagnostic or therapeutic systems that includes said robotic medical device system, said bedside interface device including user interface logic stored in the memory configured to be executed by said electronic control unit and further configured to display on said display panel a user interface which includes a first group of buttons corresponding to the plurality of diagnostic and therapeutic systems, said user interface logic is further configured to receive from the user a user touch input from said touch-responsive display panel, said user interface logic being further configured to allow the user to select one button from said first group of buttons according to said user touch input to thereby select a corresponding one of said plurality of diagnostic and therapeutic systems and to present, in response to said user selection, an application-specific user interface on said display panel that enables access to and control of said one user-selected diagnostic and therapeutic system; wherein a user-selected diagnostic and therapeutic system comprises said robotic medical device system, and wherein said user interface logic is further configured to allow the user to interact with said application-specific user interface to acquire user touch input corresponding to a command for the robotic medical device system, and wherein said electronic control unit is configured to communicate the command to the robotic medical device system. ### Claim 22 (depends on 21) The apparatus of claim 21 where said application-specific user interface of said user-selected diagnostic or therapeutic system comprises at least one application-specific second button displayed on said display panel that is different from said first group of buttons. ### Claim 23 (depends on 21) The apparatus of claim 21 wherein said user interface logic is configured to present on said display panel, for each one of said plurality of diagnostic or therapeutic systems when selected by the user, a respective application-specific user interface that enables access to and control of said user-selected diagnostic or therapeutic system. ### Claim 24 (depends on 21) The apparatus of claim 21 wherein said user interface logic is further configured to switch between respective application-specific user interfaces via a common interface displayed on said display panel, wherein said common interface includes said first group of buttons. ### Claim 25 (depends on 21) The apparatus of claim 21 wherein said electronic control unit communicates said identified command wirelessly. ### Claim 26 (depends on 21) The apparatus of claim 21 wherein said robotic medical device system includes a medical device and wherein said command is configured to cause said robotic medical device system to manipulate or maneuver the medical device. ### Claim 27 (depends on 26) The apparatus of claim 26 wherein said medical device comprises a catheter. ### Claim 28 (depends on 21) The apparatus of claim 21 wherein said robotic medical device system includes a medical device, a frame, a support structure attached to the frame, support linkages attached to the support structure, and a manipulator assembly attached to the support linkages and configured to control the positioning of the medical device. ### Claim 29 (depends on 28) The apparatus of claim 28 wherein the medical device comprises one or more tools. ### Claim 30 (depends on 28) The apparatus of claim 28 wherein the frame is supports by a plurality of wheels. ### Claim 31 (depends on 28) The apparatus of claim 28 wherein the frame is track mounted to a track for movement relative to said track. ### Claim 32 (depends on 28) The apparatus of claim 28 wherein said robotic medical device system further includes a localization system configured to determine a position of the medical device. ### Claim 33 (depends on 32) The apparatus of claim 32 wherein the position of the medical device comprises a distal tip position corresponding to the distal end portion of the medical device, said localization system is further configured to determine an orientation of the distal end portion. ### Claim 34 (depends on 32) The apparatus of claim 32 wherein the localization system further comprises a position sensor disposed in said medical device.