# US 2024/0197409 A1 · Tool navigation in mixed reality computer-assisted surgery > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0197409 A1 - Document ID: US20240197409A1 - Status: Published application - Kind: A1 - Application number: 18/541,167 - Filed: 2023-12-15 - Published: 2024-06-20 - Earliest claimed: 2022-12-20 - Assignees: Orthosoft Ulc - Applicants: Orthosoft Ulc - Inventors: Pierre COUTURE; Jerome CASAUBON; Louis-Philippe AMIOT; Jean-Sébastien MERETTE - Web page: https://patentgrep.com/patent/US20240197409A1 ## CPC - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 90/36: {Image-producing devices or illumination devices not otherwise provided for} - A61B 17/142: {with reciprocating saw blades, e.g. with cutting edges at the distal end of the saw blades} - A61B 17/157: {Cutting tibia} - A61B 17/1622: {Drill handpieces (A61B17/1626, A61B17/1628 take precedence)} - A61B 34/25: {User interfaces for surgical systems} - A61B 34/30: Surgical robots - A61B 90/11: with guides for needles or instruments, e.g. arcuate slides or ball joints - A61B 2017/568: {produced with shape and dimensions specific for an individual patient} - A61B 2034/102: {Modelling of surgical devices, implants or prosthesis} - A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2055: {Optical tracking systems} - A61B 2034/2065: {Tracking using image or pattern recognition} - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61B 2090/371: {with simultaneous use of two cameras} - A61B 2090/372: {Details of monitor hardware} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - A61B 2090/502: {Headgear, e.g. helmet, spectacles} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract A system for tracking a surgical tool relative to a bone in computer-assisted surgery, comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: tracking the surgical tool relative to the bone; merging virtual models of the surgical tool and the bone to the surgical tool and the bone in the tracking; calculating a location of a working end of the surgical tool relative to the bone using the tracking, in a concealed condition of the working end of the surgical tool relative to the bone; and outputting the location of the working end of the surgical tool relative to the bone. ## Claims ### Claim 1 A system for tracking a surgical tool relative to a bone in computer-assisted surgery, comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: tracking the surgical tool relative to the bone; merging virtual models of the surgical tool and the bone to the surgical tool and the bone in the tracking; calculating a location of a working end of the surgical tool relative to the bone using the tracking, in a concealed condition of the working end of the surgical tool relative to the bone; and outputting the location of the working end of the surgical tool relative to the bone. ### Claim 2 (depends on 1) The system according to claim 1, wherein the computer-readable program instructions are executable by the processing unit for indicating a proximity of the working end with a boundary of the bone. ### Claim 3 (depends on 1) The system according to claim 1, wherein the computer-readable program instructions are executable by the processing unit for stopping the surgical tool when the working end is at the boundary of the bone. ### Claim 4 (depends on 1) The system according to claim 1, wherein the computer-readable program instructions are executable by the processing unit for imaging and displaying the concealed working end of the surgical tool relative to the bone. ### Claim 5 (depends on 4) The system according to claim 4, wherein the computer-readable program instructions are executable by the processing unit for imaging and displaying the concealed working end of the surgical tool relative to the bone in mixed reality on a face shield worn by the operator. ### Claim 6 (depends on 1) The system according to claim 1, wherein the computer-readable program instructions are executable by the processing unit are for calibrating the surgical tool by obtaining video images of the surgical tool and its working end and processing same to size the model of the surgical tool. ### Claim 7 (depends on 6) The system according to claim 6, wherein obtaining the video images of the surgical tool and its working end includes operating the surgical tool to image an amplitude of movement of the working end and size the amplitude of movement of the working end. ### Claim 8 (depends on 1) The system according to claim 1, including obtaining the virtual model of the bone from pre-operative imaging. ### Claim 9 (depends on 1) The system according to claim 1, including obtaining the virtual model of the surgical tool from a manufacturer file. ### Claim 10 (depends on 1) The system according to claim 1, wherein the computer-readable program instructions are executable by the processing unit for controlling a robot arm as a function of a position and orientation of the tool. ### Claim 11 (depends on 10) The system according to claim 10, wherein continuously outputting the location of the tool includes continuously outputting the location of the robot arm. ### Claim 12 (depends on 1) The system according to claim 1, wherein tracking the surgical tool relative to the bone includes tracking the surgical tool relative to the bone from an image capture device at a point of view of an operator. ### Claim 13 (depends on 12) The system according to claim 12, wherein tracking the surgical tool relative to the bone includes obtaining a second feed of tracking from a stationary image capture device. ### Claim 14 (depends on 13) The system according to claim 13, wherein obtaining the video images from two different image capture devices includes prioritizing the video images from one of the image capture devices over the other of the image capture devices. ### Claim 15 (depends on 1) The system according to claim 1, wherein tracking the surgical tool relative to the bone includes obtaining video images of the surgical tool relative to the bone, and wherein merging virtual models of the surgical tool and the bone to the surgical tool and the bone in the tracking includes processing the video images. ### Claim 16 (depends on 15) The system according to claim 15, wherein calculating the location of the working end of the surgical tool relative to the bone includes using the video images processed. ### Claim 17 (depends on 16) The system according to claim 16, wherein obtaining video images includes obtaining the video images in a lower frequency capture mode when the tool is distal to the bone, and in a higher frequency capture mode when the tool is proximal to the bone. ### Claim 18 (depends on 15) The system according to claim 15, including a head-mounted device with at least one camera for obtaining the video images. ### Claim 19 (depends on 18) The system according to claim 18, wherein the head-mounted device has a display for outputting data associated with the position and orientation of the at least one object, the data being output in mixed reality. ### Claim 20 A system for tracking a surgical tool relative to a bone in computer-assisted surgery, comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining video images of the surgical tool relative to the bone, from a point of view on an operator; processing the video images to merge virtual models of the surgical tool and the bone to the surgical tool and the bone in the video images; calculating a location of a working end of the surgical tool relative to the bone using the video images processed, in a concealed condition of the working end of the surgical tool relative to the bone; and outputting the location of the working end of the surgical tool relative to the bone.