# US 2013/0096575 A1 · System and method for controlling a remote medical device guidance system in three-dimensions using gestures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0096575 A1
- Document ID: US20130096575A1
- Status: Published application
- Kind: A1
- Application number: 13/692,356
- Filed: 2012-12-03
- Published: 2013-04-18
- Earliest claimed: 2009-07-22
- Assignees: St. Jude Medical, Atrial Fibrillation Division, Inc.
- Inventors: Eric S. Olson
- Web page: https://patentgrep.com/patent/US20130096575A1

## CPC

- A61B 34/74: {Manipulators with manual electric input means}
- A61B 34/25: {User interfaces for surgical systems}
- A61B 34/76: {Manipulators having means for providing feel, e.g. force or tactile feedback}
- B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor}
- G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/03545: {Pens or stylus}
- G06T 19/003: {Navigation within 3D models or images}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- A61B 5/283: Invasive
- A61B 34/30: Surgical robots
- A61B 2034/258: {providing specific settings for specific users}
- A61B 2034/741: {Glove like input devices, e.g. "data gloves"}
- G06T 2210/41: Medical
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system for enabling a user to remotely control a robotic medical device system includes a motion capture apparatus to capture motion of a user in a sensing volume and generate indicative output data. The system includes a control unit configured to execute gesture recognition logic that recognizes a user gesture based on analysis of the indicative output data. The control unit executes interpreter logic that is configured to translate the recognized user gesture into a corresponding robotic medical device control command configured to control an aspect of the operation of the robotic medical device system.

## Claims

### Claim 1

A system for enabling a user to remotely control a robotic medical device system, comprising:
a motion capture apparatus configured to capture motion of a user in a sensing volume and generate output data indicative of the captured user motion;
an electronic control unit including a processor and a memory;
gesture recognition logic stored in said memory and configured to execute on said processor, said gesture recognition logic being configured to recognize a user gesture based on said output data from said motion capture apparatus; and
interpreter logic stored in said memory and configured to execute said processor, said interpreter logic being configured to translate the user gesture to a corresponding robotic medical device control command wherein said command is configured to control an aspect of the operation of the robotic medical device system, said electronic control unit being configured to communicate said command to the robotic medical device system.

### Claim 2 (depends on 1)

The system of claim 1 wherein said output data includes fiducial point tracking data.

### Claim 3 (depends on 2)

The system of claim 2 wherein said fiducial point tracking data is associated with a plurality of fiducial points defined with respect to the user, and wherein said fiducial point tracking data includes, for each fiducial point, a respective position.

### Claim 4 (depends on 3)

The system of claim 3 wherein each position includes a respective three-dimensional coordinate in a reference coordinate system, and wherein said fiducial point tracking data further includes, for each fiducial point, a respective time-based plurality of positions.

### Claim 5 (depends on 4)

The system of claim 4 wherein said gesture recognition logic is configured to:
identify a start pose based on said fiducial point tracking data;
record the motion of a predetermined plurality of fiducial points after the start pose until an end pose is identified based on said fiducial point tracking data;
compare said recorded motion of the predetermined plurality of fiducial points with a plurality of predefined gestures; and
output the user gesture when said recorded motion matches one of the plurality of gestures.

### Claim 6 (depends on 5)

The system of claim 5 wherein the start pose corresponds to a start condition where a first set of fiducial points assumes a first relationship therebetween, and the end pose corresponds to an end condition where a second set of fiducial points assumes a second relationship therebetween.

### Claim 7 (depends on 1)

The system of claim 1 further comprising a user-actuatable switch coupled to said electronic control unit and having a normally open state, and a user-actuatable closed state, wherein said electronic control unit is configured to at least disable said communication of said command to the robotic medical device system unless said switch is in said closed state.

### Claim 8 (depends on 1)

The system of claim 1 wherein said interpreter logic is further configured to selectively translate, based on a state of context switch parameter, the user gesture into one of (i) the robotic medical device control command configured to control an aspect of the operation of the robotic medical device system and (ii) a mapping control command configured to control an aspect of an electro-anatomic mapping system.

### Claim 9 (depends on 8)

The system of claim 8 wherein the sensing volume includes a context switching portion, said system for enabling the user to remotely control further comprising context switching logic stored in said memory and configured for execution by said processor and configured to detect a predetermined context switching gesture based on said output data from said motion capture apparatus originating with user-motion occurring in the context switching portion of the sensing volume.

### Claim 10 (depends on 3)

The system of claim 3 wherein a characteristic associated said robot catheter control command is a commanded magnitude associated with an action involving one of a catheter and a sheath under control of the robotic medical device system, and wherein the commanded magnitude corresponds to a distance between preselected fiducial points.

### Claim 11 (depends on 10)

The system of claim 10 wherein the action is selected from the group comprising a catheter extension, a catheter retraction, a sheath extension, and a sheath retraction.

### Claim 12 (depends on 3)

The system of claim 3 wherein a characteristic associated with the robot medical device control command is a commanded rotation associated with an action involving one of a catheter and a sheath under control of the robotic medical device system, and wherein the commanded rotation corresponds to a rotation angle through which a preselected fiducial point is rotated during the user gesture.

### Claim 13 (depends on 1)

The system of claim 1 wherein said output data includes acquired imaging of the movements of the user, said gesture recognition logic being configured to determine the gesture from at least said acquired imaging.

### Claim 14 (depends on 8)

The system of claim 8 said interpreter logic is further configured to generate said robotic medical device system command with respect to the visible orientation of the then-visible view of an anatomical model of at least a portion of a body of a patient produced by the mapping system.

### Claim 15 (depends on 1)

The system of claim 1 wherein said motion capture apparatus comprises an optical sub-system configured to optically detect the motion of the user in the sensing volume.

### Claim 16 (depends on 1)

The system of claim 1 wherein said motion capture apparatus comprises one of (i) a spatially detected glove configured to be worn by the user, (ii) a spatially detected wand configured to be moved by the user, and (iii) a spatially detected finger instrument configured to be worn on a finger of the user.

### Claim 17

A system for enabling a user to remotely control a robotic medical device system, comprising:
motion capture means for capturing motion of a user in a sensing volume and generating output data indicative of the captured user motion;
gesture recognition means for recognizing a user gesture based on said output data from said motion capture means;
interpreter means for translating the user gesture to a corresponding robotic medical device control command wherein said command is configured to control an aspect of the operation of the robotic medical device system; and
communication means for communicating said command to the robotic medical device system.

### Claim 18 (depends on 17)

The system of claim 17 wherein said output data includes fiducial point tracking data associated with a plurality of fiducial points defined with respect to the user, and wherein said fiducial point tracking data includes, for each fiducial point, a respective position, wherein each position includes a respective three-dimensional coordinate in a reference coordinate system, and wherein said fiducial point tracking data further includes, for each fiducial point, a respective time-based plurality of positions.

### Claim 19 (depends on 18)

The system of claim 18 wherein said gesture recognition means is further configured for:
identifying a start pose based on said fiducial point tracking data;
recording the motion of a predetermined plurality of fiducial points after the start pose until an end pose is identified based on said fiducial point tracking data;
comparing said recorded motion of the predetermined plurality of fiducial points with a plurality of predefined gestures; and
outputting the user gesture when said recorded motion matches one of the plurality of gestures.

### Claim 20

An article of manufacture, comprising:
a computer-readable storage medium having a computer program encoded thereon that, when executed by an electronic control unit, enables a user to remotely control a robotic medical device system, the computer program including code for:
capturing, using a motion capture apparatus, the motion of a user in a sensing volume and generating output data indicative of the captured user motion;
recognizing a user gesture based on said output data from the motion capture apparatus;
translating the user gesture to a corresponding robotic medical device control command wherein said command is configured to control an aspect of the operation of the robotic medical device system; and
communicating said command to the robotic medical device system.
