# US 12,178,523 B2 · Computer assisted surgical navigation system for spine procedures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,178,523 B2 - Document ID: US12178523B2 - Status: Granted - Kind: B2 - Application number: 17/656,522 - Filed: 2022-03-25 - Published: 2023-03-16 - Granted: 2024-12-31 - Earliest claimed: 2021-04-19 - Assignees: Globus Medical, Inc. - Applicants: Globus Medical, Inc. - Inventors: Douglas Legere; Norbert Johnson; Gerd Schmieta; Thomas Calloway; Dana Wisniewski; Dale Earle; Ryan Fischer - Web page: https://patentgrep.com/patent/US12178523B2 ## CPC - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations - A61B 34/25: {User interfaces for surgical systems} - A61B 34/30: Surgical robots - A61B 2017/00203: {with speech control or speech recognition} - A61B 2017/00207: {with hand gesture control or hand gesture recognition} - A61B 2034/102: {Modelling of surgical devices, implants or prosthesis} - A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones} - A61B 2034/107: {Visualisation of planned trajectories or target regions} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2055: {Optical tracking systems} - A61B 2034/207: {Divots for calibration} - A61B 2034/254: {being adapted depending on the stage of the surgical procedure} - A61B 2034/258: {providing specific settings for specific users} - A61B 2090/363: {Use of fiducial points} - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61B 2090/376: {using X-rays, e.g. fluoroscopy} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - A61B 2090/502: {Headgear, e.g. helmet, spectacles} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract A surgical system for computer assisted navigation during surgery, includes at least one processor that obtains a 3D radiological representation of a targeted anatomical structure of a patient and a set of fiducials of a registration fixture. The operations attempt to register locations of the set of fiducials in the 3D radiological representation to a 3D imaging space tracked by a camera tracking system. Based on determining one of the fiducials of the set has a location that was not successfully registered to the 3D imaging space, the operations display at least one view of the 3D radiological representation with a graphical overlay indicating the fiducial has not been successfully registered to the 3D imaging space, receive user-supplied location information identifying where the fiducial is located in the 3D radiological representation, and register the location of the fiducial to the 3D imaging space based on the user-supplied location information. ## Claims ### Claim 1 A surgical system for computer assisted navigation during surgery, the surgical system comprising: a camera tracking system including one or more cameras; a display assembly; and a housing base assembly including a processing platform having a memory, a user interface, communication circuitry configured to communicate with the camera tracking system and the display assembly, and at least one processor configured to: obtain a three-dimensional (3D) radiological representation of a targeted anatomical structure of a patient containing a set of radiopaque fiducials of a registration fixture, the 3D radiological representation being derived from a 3D scan of the patient with a radiologic imaging device; attempt to register locations of the set of fiducials contained in the 3D radiological representation to a 3D imaging space tracked by the camera tracking system; based on determining that one of the fiducials of the set has a location that was not successfully registered to the 3D imaging space, display, via the display assembly, at least one two-dimensional (2D) view of the 3D radiological representation with a graphical overlay indicating that the fiducial has not been successfully registered to the 3D imaging space, receive user-supplied location information, via the user interface, identifying where the fiducial is located in the displayed 2D view of the 3D radiological representation, and register the location of the fiducial to the 3D imaging space based on the user-supplied location information; and obtain, from at least one camera of the camera tracking system, optical images of a reference array fixated to the patient and of the registration fixture, the reference array including a first set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space, and the registration fixture including a second set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space; wherein the operation to receive user-supplied location information identifying where the fiducial is located in the 3D radiological representation, comprises to: display three orthogonal views of the fiducial in the 3D radiological representation and/or the 3D imaging space; display a graphical object overlaid on an initial location in the three orthogonal views; move location of where the graphical object is displayed in the three orthogonal views responsive to input from the user through a user interface; determine location of the fiducial in the 3D radiological representation and/or the 3D imaging space based on the location of where the graphical object is displayed in the three orthogonal views; and the operation to register the location of the fiducial to the 3D imaging space is based on the determined location of the fiducial in the 3D radiological representation; wherein the operation to display the graphical object overlaid on the initial location in the three orthogonal views, comprises: determining the initial location to correspond to a predicted location of the fiducial based on relative locations of fiducials defined by a registration fixture template. ### Claim 2 (depends on 1) The surgical system of claim 1, wherein the at least one processor is further operative to move the location where the graphical object is displayed in the three orthogonal views to track directional inputs received through the user interface of the surgical system. ### Claim 3 (depends on 1) The surgical system of claim 1, wherein the at least one processor is further operative to: track locations of a tool captured in video from a camera of the camera tracking system while the tool is being moved by a user toward one of the fiducials that was successfully registered to the 3D image space; display updated representations of the tool according to the tracked locations in the 3D imaging space; and confirm registration accuracy of the one of the fiducials that was successfully registered to the 3D image space based on comparison of a designated one of the tracked locations of the tool to the location of the one of the fiducials registered to the 3D image space. ### Claim 4 (depends on 1) The surgical system of claim 1, wherein the operation to attempt to register locations of the set of fiducials in the 3D radiological representation to the 3D imaging space tracked by the camera tracking system, comprises: obtain, from at least one camera of the camera tracking system, an optical image of a reference array fixated to the patient, the reference array including a set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space; attempt to register locations of a pattern of the set of optical markers to locations of a pattern of the set of fiducials in the 3D radiological representation; and identify any of the optical markers of the reference array that are not successfully registered to any of the fiducials of the registration fixture. ### Claim 5 (depends on 1) The surgical system of claim 1, wherein the operations further comprise to: display a virtual implant device as an overlay on a view of the 3D radiological representation of the targeted anatomical structure; display a graphical indication of a trajectory of the virtual implant device representing an implantation trajectory of the virtual implant device into the targeted anatomical structure; update pose of the graphical indication of the trajectory of the virtual implant device displayed in the view of the 3D radiological representation, to track steering inputs received through a user interface of the surgical system; and store as a planned trajectory of the virtual implant device, a user-designated one of the poses of the graphical indication of the trajectory. ### Claim 6 (depends on 5) The surgical system of claim 5, wherein the operations further comprise to: display a set of implant devices which are selectable by a user for implant planning; and generate a graphical representation of the virtual implant device based on a template of one of the set of user-selectable implant devices which is selected by a user through the user interface. ### Claim 7 (depends on 5) The surgical system of claim 5, wherein the operations further comprise to: display the planned trajectory of virtual implant device as an overlay on the view of the 3D radiological representation of the targeted anatomical structure; obtain, from at least one camera of the camera tracking system, optical images of a reference array fixated to a real-implant device corresponding to the virtual implant device, the reference array including a set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space; track pose of the real-implant device in the 3D imaging space based on pose of the reference array in the optical images while the real implant device is being positioned by a user relative to the targeted anatomical structure of the patient; and display updated graphical representations of the real-implant device relative to the planned trajectory of the virtual implant device according to the tracked pose in the 3D imaging space. ### Claim 8 (depends on 1) The surgical system of claim 1, wherein prior to obtaining the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture, the at least one processor is further operative to: obtain a fluoroscopic image of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture; determine whether a first condition is satisfied based on a defined number of the optical markers in the first set being detected by the at least one camera of the camera tracking system in the 3D imaging space; determine whether a second condition is satisfied based on a defined number of the optical markers in the second set being detected by the at least one camera of the camera tracking system in the 3D imaging space; determine whether a third condition is satisfied based on a defined number of the set of fiducials of the registration fixture being visible in the fluoroscopic image; when one of the first, second, and third conditions is not satisfied, display an indication of the not satisfied one of the first, second, and third conditions, and inhibit capture by a radiological imaging process of the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture; and when each of the first, second, and third conditions are satisfied, enable capture by the radiological imaging process of the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture. ### Claim 9 (depends on 8) The surgical system of claim 8, wherein the at least one processor is further operative to: trigger capture of anteroposterior and lateral fluoroscopic images at a plurality of defined locations of the targeted anatomical structure of the patient based on determining each of the first, second, and third conditions are satisfied; and compute the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture based on the captured anteroposterior and lateral fluoroscopic images at the plurality of defined locations of the targeted anatomical structure of the patient. ### Claim 10 A method by a surgical system for computer assisted navigation during surgery, the method comprising: providing the surgical system for computer assisted navigation during surgery, the system including: a camera tracking system including one or more cameras; a display assembly; and a housing base assembly including a processing platform having a memory, a user interface, communication circuitry configured to communicate with the camera tracking system and the display assembly, and at least one processor; obtaining a three-dimensional (3D) radiological representation of a targeted anatomical structure of a patient containing a set of fiducials of a registration fixture; attempting to register locations of the set of fiducials contained in the 3D radiological representation to a 3D imaging space tracked by the camera tracking system; based on determining that one of the fiducials of the set has a location that was not successfully registered to the 3D imaging space, displaying, via the display device, at least one two-dimensional (2D) view of the 3D radiological representation with a graphical overlay indicating that the fiducial has not been successfully registered to the 3D imaging space, receiving user-supplied location information, via the user interface, identifying where the fiducial is located in the displayed 2D view of the 3D radiological representation, and registering the location of the fiducial to the 3D imaging space based on the user-supplied location information; and obtaining, from at least one camera of the camera tracking system, optical images of a reference array fixated to the patient and of the registration fixture, the reference array including a first set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space, and the registration fixture including a second set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space; wherein: the receiving user-supplied location information identifying where the fiducial is located in the 3D radiological representation, comprises: displaying three orthogonal views of the fiducial in the 3D radiological representation and/or the 3D imaging space; displaying a graphical object overlaid on an initial location in the three orthogonal views; moving location of where the graphical object is displayed in the three orthogonal views responsive to input from the user through a user interface; determining location of the fiducial in the 3D radiological representation and/or the 3D imaging space based on the location of where the graphical object is displayed in the three orthogonal views; and the registering the location of the fiducial to the 3D imaging space is performed based on the determined location of the fiducial in the 3D radiological representation; wherein the operation to display the graphical object overlaid on the initial location in the three orthogonal views, comprises: determining the initial location to correspond to a predicted location of the fiducial based on relative locations of fiducials defined by a registration fixture template. ### Claim 11 (depends on 10) The method of claim 10, further comprising: tracking locations of a tool captured in video from a camera of the camera tracking system while the tool is being moved by a user toward one of the fiducials that was successfully registered to the 3D image space; displaying updated representations of the tool according to the tracked locations in the 3D imaging space; and confirming registration accuracy of the one of the fiducials that was successfully registered to the 3D image space based on comparison of a designated one of the tracked locations of the tool to the location of the one of the fiducials registered to the 3D image space. ### Claim 12 (depends on 10) The method of claim 10, wherein the attempting to register locations of the set of fiducials in the 3D radiological representation to the 3D imaging space tracked by the camera tracking system, comprises: obtaining, from at least one camera of the camera tracking system, an optical image of a reference array fixated to the patient, the reference array including a set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space; attempting to register locations of a pattern of the set of optical markers to locations of a pattern of the set of fiducials in the 3D radiological representation; and identifying any of the optical markers of the reference array that are not successfully registered to any of the fiducials of the registration fixture. ### Claim 13 (depends on 10) The method of claim 10, further comprising: displaying a virtual implant device as an overlay on a view of the 3D radiological representation of the targeted anatomical structure; displaying a graphical indication of a trajectory of the virtual implant device representing an implantation trajectory of the virtual implant device into the targeted anatomical structure; updating pose of the graphical indication of the trajectory of the virtual implant device displayed in the view of the 3D radiological representation, to track steering inputs received through a user interface of the surgical system; and storing as a planned trajectory of the virtual implant device, a user-designated one of the poses of the graphical indication of the trajectory. ### Claim 14 (depends on 10) The method of claim 10, further comprising: displaying a set of implant devices which are selectable by a user for implant planning; and generating a graphical representation of the virtual implant device based on a template of one of the set of user-selectable implant devices which is selected by a user through the user interface. ### Claim 15 (depends on 10) The method of claim 10, further comprising: displaying the planned trajectory of virtual implant device as an overlay on the view of the 3D radiological representation of the targeted anatomical structure; obtaining, from at least one camera of the camera tracking system, optical images of a reference array fixated to a real-implant device corresponding to the virtual implant device, the reference array including a set of optical markers detectable by the at least one camera of the camera tracking system in the 3D imaging space; tracking pose of the real-implant device in the 3D imaging space based on pose of the reference array in the optical images while the real implant device is being positioned by a user relative to the targeted anatomical structure of the patient; and displaying updated graphical representations of the real-implant device relative to the planned trajectory of the virtual implant device according to the tracked pose in the 3D imaging space. ### Claim 16 (depends on 10) The method of claim 10, further comprising, prior to obtaining the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture: obtaining a fluoroscopic image of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture; determining whether a first condition is satisfied based on a defined number of the optical markers in the first set being detected by the at least one camera of the camera tracking system in the 3D imaging space; determining whether a second condition is satisfied based on a defined number of the optical markers in the second set being detected by the at least one camera of the camera tracking system in the 3D imaging space; determining whether a third condition is satisfied based on a defined number of the set of fiducials of the registration fixture being visible in the fluoroscopic image; when one of the first, second, and third conditions is not satisfied, displaying an indication of the not satisfied one of the first, second, and third conditions, and inhibiting capture by a radiological imaging process of the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture; and when each of the first, second, and third conditions are satisfied, enabling capture by the radiological imaging process of the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture. ### Claim 17 (depends on 16) The method of claim 16, further comprising: triggering capture of anteroposterior and lateral fluoroscopic images at a plurality of defined locations of the targeted anatomical structure of the patient based on determining each of the first, second, and third conditions are satisfied; and computing the 3D radiological representation of the targeted anatomical structure of the patient and the set of fiducials of the registration fixture based on the captured anteroposterior and lateral fluoroscopic images at the plurality of defined locations of the targeted anatomical structure of the patient.