# US 2019/0167366 A1 · Systems and methods to correct for uncommanded instrument roll > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0167366 A1 - Document ID: US20190167366A1 - Status: Published application - Kind: A1 - Application number: 16/210,971 - Filed: 2018-12-05 - Published: 2019-06-06 - Earliest claimed: 2017-12-06 - Assignees: Auris Health, Inc. - Applicants: Auris Health, Inc. - Inventors: Ritwik Ummalaneni; Allen Jiang - Web page: https://patentgrep.com/patent/US20190167366A1 ## CPC - A61B 34/37: Leader-follower robots (A61B34/35 takes precedence) - A61B 90/37: {Surgical systems with images on a monitor during operation} - G06T 7/248: {involving reference images or patches} - G06T 7/73: using feature-based methods - G06T 7/74: {involving reference images or patches} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2065: {Tracking using image or pattern recognition} - G06T 2207/10016: Video; Image sequence - G06T 2207/10068: Endoscopic image - G06T 2207/30004: Biomedical image processing - G06T 2207/30196: Human being; Person - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Certain aspects relate to systems, methods, and techniques for correcting for uncommanded instrument roll. A method for adjusting a controller-feedback system in a medical instrument may comprise receiving data from a sensor at or near a distal end of the instrument, determining a tip frame of reference based on the data from the sensor, the tip frame of reference representing a current orientation of the distal end of the instrument, obtaining a desired frame of reference, determining an adjustment to a visual frame of reference or a control frame of reference based on the tip frame of reference and the desired frame of reference, and transforming the visual frame of reference or the control frame of reference. ## Claims ### Claim 1 A method for adjusting a controller-feedback system in a robotically controlled medical instrument inserted into a patient, the method comprising: receiving data from at least one imaging device or location sensor at a distal end of the instrument; determining a tip frame of reference based on the data from the at least one imaging device or location sensor, the tip frame of reference representing a current orientation of the distal end of the instrument; obtaining a desired frame of reference representing a frame of reference relative to an anatomy frame of reference or a global frame of reference; determining an adjustment to a visual frame of reference or a control frame of reference based on the tip frame of reference and the desired frame of reference; and transforming the visual frame of reference or the control frame of reference based on the determined adjustment. ### Claim 2 (depends on 1) The method of claim 1, wherein obtaining the desired frame of reference is based on one or more pixel values of an image representing one or more anatomical features of the patient. ### Claim 3 (depends on 1) The method of claim 1, wherein obtaining the desired frame of reference is based on one or more anatomical features of a main carina of the patient. ### Claim 4 (depends on 1) The method of claim 1, wherein obtaining the desired frame of reference is based on data from one or more EM patches positioned on the patient. ### Claim 5 (depends on 1) The method of claim 1, wherein obtaining the desired frame of reference is based on the tip frame of reference. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the adjustment is based on comparing between one or more features derived from at least one image of an anatomical feature and one or more features derived from at least one model of the anatomical feature. ### Claim 7 (depends on 6) The method of claim 6, wherein the at least one image of an anatomical feature is obtained from the imaging device at the distal end of the instrument. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the adjustment is based on comparing between data from one or more EM sensors in the distal end of the instrument and data from one or more EM patches positioned on the patient. ### Claim 9 (depends on 1) The method of claim 1, wherein determining the adjustment is based on data from an accelerometer configured to measure force of gravity. ### Claim 10 (depends on 1) The method of claim 1, wherein transforming the visual frame of reference or the control frame of reference comprises rotating the visual frame of reference or the control frame of reference with respect to a longitudinal axis of the instrument. ### Claim 11 (depends on 1) The method of claim 1, wherein transforming the visual frame of reference or the control frame of reference comprises rotating the visual frame of reference or the control frame of reference to align with the tip frame of reference or the desired frame of reference. ### Claim 12 (depends on 1) The method of claim 1, wherein transforming the visual frame of reference or the control frame of reference is based on a user input. ### Claim 13 (depends on 1) The method of claim 1, further comprising verifying the transformed visual frame of reference or the transformed control frame of reference. ### Claim 14 (depends on 13) The method of claim 13, wherein verifying the transformed visual frame of reference or the transformed control frame of reference comprises: moving the instrument in one direction; calculating an expected change in the visual frame of reference or the control frame of reference in response to the movement of the instrument; and comparing between (1) an actual change in the visual frame of reference or the control frame of reference and (2) the expected change. ### Claim 15 A system configured to transform a control frame of a medical instrument configured to be inserted into a patient, the system comprising: a control system configured to determine movement of the medical instrument; at least one computer-readable memory having stored thereon executable instructions; and one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the system to at least: obtain a control frame of reference representing relationship between a motor control command and a motor output of the medical instrument; determine a tip frame of reference based on data from at least one imaging device or location sensor at a distal end of the medical instrument, the tip frame of reference representing a current orientation of the distal end of the medical instrument; obtain a desired frame of reference; and transform the control frame of reference based on the tip frame of reference and the desired frame of reference. ### Claim 16 (depends on 15) The system of claim 15, wherein the at least one location sensor comprises an electromagnetic (EM) sensor. ### Claim 17 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: receive the tip frame of reference from the medical instrument. ### Claim 18 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: (1) receive data from a second sensor at the distal end of the medical instrument and (2) determine the tip frame of reference based on the data from the second sensor. ### Claim 19 (depends on 18) The system of claim 18, wherein the second sensor at the distal end of the instrument comprises at least one imaging device or location sensor. ### Claim 20 (depends on 19) The system of claim 19, wherein the second sensor comprises an electromagnetic (EM) sensor. ### Claim 21 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: receive the control frame of reference from the control system. ### Claim 22 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: (1) move the medical instrument in one direction and (2) determine the control frame of reference based on the movement of the medical instrument. ### Claim 23 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: receive a visual frame of reference from the instrument and transform the visual frame of reference based on the tip frame of reference and the desired frame of reference. ### Claim 24 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: (1) receive data from the at least one sensor at the distal end of the medical instrument, (2) determine a visual frame of reference based on the data from the at least one sensor, and (3) transform the visual frame of reference based on the tip frame of reference and the desired frame of reference. ### Claim 25 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: determine the desired frame of reference based on one or more anatomical features of the patient. ### Claim 26 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: determine the desired frame of reference based on one or more pixel values of an image representing the one or more anatomical features of the patient. ### Claim 27 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: determine the desired frame of reference based on data from one or more EM patches positioned on the patient. ### Claim 28 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: determine one or more differences between at least one image of an anatomical feature and at least one model of the anatomical feature; and transform the control frame of reference based on the differences between the at least one image and the at least one model. ### Claim 29 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: rotate the control frame of reference with respect to a longitudinal axis of the medical instrument to align with the desired frame of reference. ### Claim 30 (depends on 15) The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least: transform the control frame of reference based on a user input.