# US 11,992,282 B2 · Motion capture controls for robotic surgery > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,992,282 B2 - Document ID: US11992282B2 - Status: Granted - Kind: B2 - Application number: 16/354,422 - Filed: 2019-03-15 - Published: 2020-09-17 - Granted: 2024-05-28 - Earliest claimed: 2019-03-15 - Assignees: Cilag GmbH International - Applicants: Cilag GmbH International - Inventors: Clinton W. Denlinger; Charles J. Scheib; Jeffrey S. Swayze - Web page: https://patentgrep.com/patent/US11992282B2 ## CPC - A61B 34/35: for telesurgery - A61B 34/37: Leader-follower robots (A61B34/35 takes precedence) - A61B 34/74: {Manipulators with manual electric input means} - A61B 34/76: {Manipulators having means for providing feel, e.g. force or tactile feedback} - A61B 34/77: {Manipulators with motion or force scaling} - A61B 90/361: {Image-producing devices, e.g. surgical cameras} - A61B 90/37: {Surgical systems with images on a monitor during operation} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2059: {Mechanical position encoders} - A61B 2034/305: {Details of wrist mechanisms at distal ends of robotic arms} - A61B 2034/742: {Joysticks} - A61B 2034/743: {Keyboards} - A61B 2034/744: {Mouse} - A61B 2090/064: {for measuring force, pressure or mechanical tension} - A61B 2090/066: {for measuring torque} - A61B 2090/371: {with simultaneous use of two cameras} - A61B 2090/373: {using light, e.g. by using optical scanners} - A61B 2090/378: {using ultrasound} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract An input control device is disclosed. The input control device includes a central portion coupled to a multi-axis force and torque sensor, which is configured to receive input control motions from a surgeon. The central portion is flexibly supported on a base. The input control device also includes a rotary joint coupled to a rotary sensor. The input control device is configured to provide control motions to a robotic arm and/or a robotic tool based on input controls detected by the multi-axis force and torque sensor and the rotary sensor. ## Claims ### Claim 1 A control system for a robotic surgical tool, the control system comprising: an untethered handpiece, comprising: a body; a joystick extending from the body; a rotatable shaft extending from the body; and a plurality of sensors comprising a body sensor embedded in the body and configured to detect motion of the body in three-dimensional space, a multi-axis force sensor configured to detect forces applied to the joystick, and a shaft sensor configured to detect rotary displacement of the shaft relative to the body; and a control circuit communicatively coupled to the plurality of sensors, a proximity detection system, and a tissue identification system, the control circuit configured to: receive, intraoperatively, imaging signals from the tissue identification system; process, intraoperatively, the imaging signals to intraoperatively identify an obscured tissue boundary, wherein the obscured tissue boundary is obscured from a clinician's view; receive, intraoperatively, proximity signals from the proximity detection system, wherein the proximity signals are indicative of a distance of the robotic surgical tool to the obscured tissue boundary; receive, intraoperatively, input control signals from the plurality of sensors; switch between a gross motion mode and a precision motion mode in response to receiving a proximity signal indicating the distance of the robotic surgical tool to the obscured tissue boundary being reduced to less than a threshold distance; provide gross motion control signals to the robotic surgical tool based on the input control signals from the multi-axis force sensor in the gross motion mode; and provide precision motion control signals to the robotic surgical tool based on the input control signals from the body sensor and the shaft sensor in the precision motion mode, wherein the input control signals in the gross motion mode are applied separate from the input control signals in the precision motion mode. ### Claim 2 (depends on 1) The control system of claim 1, wherein the joystick comprises a loop dimensioned and positioned to receive a user's thumb. ### Claim 3 (depends on 1) The control system of claim 1, wherein the body sensor is selected from at least one of an inertial sensor or an electromagnetic tracking receiver. ### Claim 4 (depends on 1) The control system of claim 1, wherein the shaft sensor is selected from at least one of a rotary transducer, strain gauge, or an optical sensor. ### Claim 5 (depends on 1) The control system of claim 1, wherein the proximity signals are indicative of a distance of the robotic surgical tool to a critical structure below the surface of the tissue. ### Claim 6 A control system for a robotic surgical tool, the control system comprising: an untethered handpiece, comprising: a body; an actuator extending from the body; a rotatable shaft extending from the body; and a plurality of sensors comprising a body sensor embedded in the body and configured to detect motion of the body in three-dimensional space, a force sensor configured to detect forces applied to the actuator, and a shaft sensor configured to detect rotary displacement of the shaft relative to the body; a tissue identification system configured to identify an obscured tissue boundary; a proximity detection system configured to intraoperatively detect a distance of the robotic surgical tool to the obscured tissue boundary; and a control circuit communicatively coupled to the plurality of sensors, the tissue identification system, and the proximity detection system, the control circuit configured to: receive, intraoperatively, imaging signals from the tissue identification system; process, intraoperatively, the imaging signals to intraoperatively identify an obscured tissue boundary, wherein the obscured tissue boundary is obscured from a clinician's view; receive, intraoperatively, proximity signals from the proximity detection system, wherein the proximity signals are indicative of the distance of the robotic surgical tool to the obscured tissue boundary; receive, intraoperatively, input control signals from the plurality of sensors; switch between a first mode and a second mode in response to receiving a proximity signal indicating the distance of the robotic surgical tool to the obscured tissue boundary being reduced to less than a threshold distance; provide first motion control signals to the robotic surgical tool based on the input control signals from the force sensor in the first mode, wherein the first motion control signals are scaled based on the proximity signal; and provide second motion control signals to the robotic surgical tool based on the input control signals from the body sensor and the shaft sensor in the second mode. ### Claim 7 (depends on 6) The control system of claim 6, wherein the proximity detection system comprises a structured light source and an optical receiver. ### Claim 8 (depends on 6) The control system of claim 6, wherein the shaft further comprises a radial sensor configured to detect radial forces applied to the shaft, and wherein the control circuit is further configured to: receive the input control signals from the radial sensor; and provide the second motion control signals to the robotic surgical tool based on the input control signals received from the radial sensor, wherein the second motion control signals control closure motions to one or more than one jaw of the robotic surgical tool. ### Claim 9 A control system for a robotic surgical tool, the control system comprising: an untethered handpiece, comprising: a body; an actuation arm extending proximally from the body; a shaft extending distally from the body; and a plurality of sensors comprising a body sensor embedded in the body and configured to detect motion of the body in three-dimensional space, an arm sensor configured to detect forces applied to the actuation arm, and a shaft sensor configured to detect rotary displacement of the shaft relative to the body; and a control circuit communicatively coupled to the plurality of sensors, a tissue identification system, and a proximity detection system, the control circuit configured to: receive, intraoperatively, imaging signals from the tissue identification system; process, intraoperatively, the imaging signals to intraoperatively identify an obscured tissue boundary, wherein the obscured tissue boundary is obscured from a clinician's view; receive, intraoperatively, proximity signals from the proximity detection system, wherein the proximity signals are indicative of a distance of the robotic surgical tool to the obscured tissue boundary; receive, intraoperatively, input control signals from the plurality of sensors; switch between a gross motion mode and a precision motion mode in response to receiving a proximity signal indicating the distance of the robotic surgical tool to the obscured tissue boundary being reduced to less than a threshold distance; provide first output control signals to the robotic surgical tool based on the input control signals from the arm sensor in the gross motion mode; and provide second output control signals to the robotic surgical tool based on the input control signals from the body sensor and the shaft sensor in the precision motion mode. ### Claim 10 (depends on 9) The control system of claim 9, wherein the actuation arm comprises a joystick movably coupled to the body at a space joint. ### Claim 11 (depends on 10) The control system of claim 10, wherein the joystick comprises a ring dimensioned and positioned to receive a user's thumb. ### Claim 12 (depends on 10) The control system of claim 10, wherein the arm sensor comprises a multi-axis force sensor positioned at the space joint. ### Claim 13 (depends on 9) The control system of claim 9, wherein the body sensor comprises an inertial sensor. ### Claim 14 (depends on 9) The control system of claim 9, wherein the body sensor comprises an electromagnetic tracking receiver. ### Claim 15 (depends on 9) The control system of claim 9, wherein the proximity detection system comprises a structured light source. ### Claim 16 (depends on 9) The control system of claim 9, wherein the control circuit is further configured to scale the first output control signals based on the input control signals received by the arm sensor in the gross motion mode in response to the proximity signals received from the proximity detection system. ### Claim 17 (depends on 16) The control system of claim 16, wherein the control circuit is further configured to reduce the first output control signals in the gross motion mode in response to the proximity signals indicating the robotic surgical tool is approaching tissue. ### Claim 18 (depends on 9) The control system of claim 9, wherein the untethered handpiece further comprises a lock for the actuation arm, wherein the lock is movable from an unlocked position to a locked position in response to receiving the proximity signal indicating the distance of the robotic surgical tool to the obscured tissue boundary being reduced to less than the threshold distance. ### Claim 19 (depends on 9) The control system of claim 9, wherein the body further comprises a spring-loaded actuation button movable between an initial position and a depressed position, and wherein the output control signals based on the input control signals detected by the body sensor in the gross motion mode are only provided to the robotic surgical tool when the spring-loaded actuation button is moved to the depressed position. ### Claim 20 (depends on 9) The control system of claim 9, wherein the shaft further comprises a radial sensor configured to detect radial forces applied to the shaft, and wherein the control circuit is further configured to: receive the input control signals from the radial sensor; and provide the second output control signals to the robotic surgical tool based on the input control signals received from the radial sensor to apply closure motions to one or more than one jaw of the robotic surgical tool. ### Claim 21 (depends on 9) The control system of claim 9, wherein the untethered handpiece further comprises a trigger and a trigger sensor configured to detect an actuation of the trigger, and wherein the control circuit is further configured to: receive the input control signals from the trigger sensor; and provide the second output control signals to the robotic surgical tool based on the input control signals from the trigger sensor to apply closure motions to one or more than one jaw of the robotic surgical tool. ### Claim 22 A control system for a robotic surgical tool, the control system comprising: an untethered handpiece, comprising: a gross motion controller comprising a multi-axis sensor; a precision motion controller comprising an embedded motion sensor; and a control circuit communicatively coupled to the multi-axis sensor, the embedded motion sensor, a tissue identification system, and a proximity detection system, the control circuit configured to: receive, intraoperatively, imaging signals from the tissue identification system; process, intraoperatively, the imaging signals to intraoperatively identify an obscured tissue boundary, wherein the obscured tissue boundary is obscured from a clinician's view; receive, intraoperatively, proximity signals from the proximity detection system indicative of a distance of the robotic surgical tool to the obscured tissue boundary; and switch between a gross motion mode, in which input control signals from the gross motion controller are utilized to control the robotic surgical tool, and a precision motion mode, in which input control signals from the precision motion controller are utilized to control the robotic surgical tool, in response to receiving a proximity signal indicating the distance of the surgical tool to the obscured tissue boundary being reduced to less than a threshold distance.