# US 2023/0056674 A1 · Optical And Non-Optical Sensor Tracking of a Robotically Controlled Instrument > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0056674 A1 - Document ID: US20230056674A1 - Status: Published application - Kind: A1 - Application number: 17/982,830 - Filed: 2022-11-08 - Published: 2023-02-23 - Earliest claimed: 2012-09-26 - Assignees: Stryker Corporation - Applicants: Stryker Corporation - Inventors: Chunwu Wu - Web page: https://patentgrep.com/patent/US20230056674A1 ## CPC - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 5/064: {using markers (A61B5/062 takes precedence)} - A61B 34/30: Surgical robots - A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers} - A61B 2017/00123: {and automatic shutdown} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2053: {Tracking an applied voltage gradient} - A61B 2034/2055: {Optical tracking systems} - A61B 2090/0818: {Redundant systems, e.g. using two independent measuring systems and comparing the signals} - A61B 2090/3945: {Active visible markers, e.g. light emitting diodes} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - Y10S 901/47: Optical - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract Surgical systems, navigation systems, and methods involving a robotic manipulator configured to control movement of an instrument to facilitate a surgical procedure. The navigation system includes a camera unit configured to optically track a pose of the instrument and a non-optical sensor coupled to the instrument. The navigation system includes a computing system coupled to the camera unit and being configured to obtain readings from the non-optical sensor. The computing system detects a condition whereby the camera unit is blocked from optically tracking the pose of the instrument. In response to detection of the condition, the computing system tracks the pose of the instrument with the readings from the non-optical sensor. ## Claims ### Claim 1 A surgical system, comprising: a robotic manipulator; an instrument coupled to the robotic manipulator, wherein the robotic manipulator is configured to control movement of the instrument to facilitate a surgical procedure; a non-optical sensor coupled to the instrument; and a navigation system configured obtain readings from the non-optical sensor, and the navigation system comprising a camera unit configured to optically track a pose of the instrument, wherein the navigation system is configured to: detect a condition whereby the camera unit is blocked from optically tracking the pose of the instrument; and in response to detection of the condition, track the pose of the instrument with the readings from the non-optical sensor. ### Claim 2 (depends on 1) The surgical system of claim 1, wherein the navigation system is configured to: detect that the camera unit is no longer blocked from optically tracking the pose of the instrument; and in response, optically track the pose of the instrument with the camera unit. ### Claim 3 (depends on 1) The surgical system of claim 1, wherein, in response to detection of the condition, the navigation system is configured to utilize readings from the non-optical sensor to estimate the pose of the instrument. ### Claim 4 (depends on 1) The surgical system of claim 1, wherein, in response to detection of the condition, the navigation system tracks the pose of the instrument by further being configured to: obtain a previous pose of the instrument that was measured based on optically tracking the pose of the instrument by the camera unit prior to detection of the condition; and utilize readings from the non-optical sensor to calculate a current pose of the instrument relative to the previous pose. ### Claim 5 (depends on 1) The surgical system of claim 1, wherein the non-optical sensor comprises an accelerometer. ### Claim 6 (depends on 1) The surgical system of claim 1, wherein the non-optical sensor comprises a gyroscope. ### Claim 7 (depends on 1) The surgical system of claim 1, wherein the robotic manipulator is configured to utilize the instrument to prepare a patient anatomy to receive an implant. ### Claim 8 (depends on 1) The surgical system of claim 1, wherein the robotic manipulator comprises a robotic arm that is supported by a moveable cart. ### Claim 9 (depends on 1) The surgical system of claim 1, wherein, in response to detection of the condition, the navigation system is configured to communicate the tracked pose of the instrument to a control system coupled to the robotic manipulator to enable the robotic manipulator to control or constrain movement of the instrument relative to a predefined path or boundary. ### Claim 10 A navigation system configured to be utilized with a robotic manipulator configured to control movement of an instrument to facilitate a surgical procedure, the navigation system comprising: a camera unit configured to optically track a pose of the instrument; a non-optical sensor coupled to the instrument; and a computing system coupled to the camera unit and being configured to obtain readings from the non-optical sensor, and the computing system being configured to: detect a condition whereby the camera unit is blocked from optically tracking the pose of the instrument; and in response to detection of the condition, track the pose of the instrument with the readings from the non-optical sensor. ### Claim 11 (depends on 10) The navigation system of claim 10, wherein the computing system is configured to: detect that the camera unit is no longer blocked from optically tracking the pose of the instrument; and in response, optically track the pose of the instrument with the camera unit. ### Claim 12 (depends on 10) The navigation system of claim 10, wherein, in response to detection of the condition, the computing system is configured to utilize readings from the non-optical sensor to estimate the pose of the instrument. ### Claim 13 (depends on 10) The navigation system of claim 10, wherein, in response to detection of the condition, the computing system tracks the pose of the instrument by further being configured to: obtain a previous pose of the instrument that was measured based on optically tracking the pose of the instrument by the camera unit prior to detection of the condition; and utilize readings from the non-optical sensor to calculate a current pose of the instrument relative to the previous pose. ### Claim 14 (depends on 10) The navigation system of claim 10, wherein the non-optical sensor comprises an accelerometer. ### Claim 15 (depends on 10) The navigation system of claim 10, wherein the non-optical sensor comprises a gyroscope. ### Claim 16 (depends on 10) The navigation system of claim 10, wherein, in response to detection of the condition, the computing system is configured to communicate the tracked pose of the instrument to a control system coupled to the robotic manipulator to enable the robotic manipulator to control or constrain movement of the instrument relative to a predefined path or boundary. ### Claim 17 A method of operating a navigation system including a camera unit, a non-optical sensor, and a computing system, the non-optical sensor being coupled to an instrument that is controlled by a robotic manipulator, the method comprising: optically tracking a pose of the instrument with the camera unit; detecting, with the computing system, a condition whereby the camera unit is blocked from optically tracking the pose of the instrument; obtaining, with the computing system, readings from the non-optical sensor; and in response to detecting the condition, the computing system tracking the pose of the instrument with the readings from the non-optical sensor. ### Claim 18 (depends on 17) The method of claim 17, comprising: detecting, with the computing system, that the camera unit is no longer blocked from optically tracking the pose of the instrument; and in response, optically tracking the pose of the instrument with the camera unit. ### Claim 19 (depends on 17) The method of claim 17, comprising: in response to detecting the condition, the computing system utilizing readings from the non-optical sensor for estimating the pose of the instrument. ### Claim 20 (depends on 17) The method of claim 17, comprising, in response to detecting the condition, the computing system tracking the pose of the instrument by further: obtaining a previous pose of the instrument that was measured based on optically tracking the pose of the instrument by the camera unit prior to detecting the condition; and utilizing readings from the non-optical sensor for calculating a current pose of the instrument relative to the previous pose.