# US 10,499,997 B2 · Systems and methods for surgical navigation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,499,997 B2 - Document ID: US10499997B2 - Status: Granted - Kind: B2 - Application number: 15/860,057 - Filed: 2018-01-02 - Published: 2018-07-05 - Granted: 2019-12-10 - Earliest claimed: 2017-01-03 - Assignees: MAKO Surgical Corp. - Applicants: MAKO Surgical Corp. - Inventors: Jeremy Weinstein; Andrei Danilchenko; Jose Luis Moctezuma de la Barrera - Web page: https://patentgrep.com/patent/US10499997B2 ## CPC - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 90/361: {Image-producing devices, e.g. surgical cameras} - A61F 2/461: {of knees} - A61B 34/30: Surgical robots - A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones} - A61B 2034/107: {Visualisation of planned trajectories or target regions} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2055: {Optical tracking systems} - A61B 2034/2057: {Details of tracking cameras} - A61B 2034/2059: {Mechanical position encoders} - A61B 2034/2065: {Tracking using image or pattern recognition} - A61B 2034/2068: {using pointers, e.g. pointers having reference marks for determining coordinates of body points} - A61B 2034/2072: {Reference field transducer attached to an instrument or patient} - A61B 2034/252: {indicating steps of a surgical procedure} - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61B 2090/368: {changing the image on a display according to the operator's position} - A61B 2090/372: {Details of monitor hardware} - A61B 2090/373: {using light, e.g. by using optical scanners} - A61B 2090/395: {with marking agent for marking skin or other tissue} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - A61B 2090/502: {Headgear, e.g. helmet, spectacles} - A61F 2002/4633: {for selection of endoprosthetic joints or for pre-operative planning} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C) ## Abstract System and methods for surgical navigation providing mixed reality visualization are provided. The mixed reality visualization depicts virtual images in conjunction with real objects to provide improved visualization to users. ## Claims ### Claim 1 A method of calibrating registration of a HMD coordinate system and a localizer coordinate system, the HMD coordinate system being associated with a head-mounted display and the localizer coordinate system being associated with a surgical navigation localizer, said method comprising the steps of: obtaining a registration of the HMD coordinate system and the localizer coordinate system such that images displayed by the head-mounted display can be associated with real objects tracked by the localizer; indicating a registration error to a user using at least one real calibration marker viewable by the user and at least one virtual calibration marker image displayed to the user, wherein the at least one virtual calibration marker image has a congruency with the at least one real calibration marker so that the at least one virtual calibration marker image is capable of being aligned with the at least one real calibration marker whereby a magnitude of misalignment is indicative of the registration error; and calibrating the registration of the HMD coordinate system and the localizer coordinate system to reduce the registration error by receiving input from the user associated with adjusting a position of the at least one virtual calibration marker image relative to the at least one real calibration marker to better align the at least one virtual calibration marker image with the at least one real calibration marker. ### Claim 2 (depends on 1) The method of claim 1, wherein the at least one real calibration marker is tracked by the localizer. ### Claim 3 (depends on 1) The method of claim 1, wherein at least one real object is tracked by the localizer, the at least one real object having a predetermined spatial relationship to the at least one real calibration marker. ### Claim 4 (depends on 1) The method of claim 1, wherein the method is started in response to the user directing a viewing direction of the user toward the real calibration markers, wherein the head-mounted display is configured to be worn by the user. ### Claim 5 (depends on 1) The method of claim 1, wherein receiving input from the user comprises receiving input that defines coordinates of the at least one virtual calibration marker image in the localizer coordinate system. ### Claim 6 A method of calibrating registration of a HMD coordinate system and a localizer coordinate system, the HMD coordinate system being associated with a head-mounted display and the localizer coordinate system being associated with a surgical navigation localizer, said method comprising the steps of: registering the HMD coordinate system and the localizer coordinate system such that images displayed by the head-mounted display can be associated with real objects tracked by the localizer; indicating a registration error to the user using a plurality of real calibration markers viewable by the user and a plurality of virtual calibration marker images displayed to the user, wherein the virtual calibration marker images have a congruency with the real calibration markers so that the virtual calibration marker images are capable of being aligned with the real calibration markers whereby a magnitude of misalignment is indicative of the registration error; and calibrating the registration of the HMD coordinate system and the localizer coordinate system to reduce the registration error by receiving input from the user associated with adjusting positions of the virtual calibration marker images relative to the real calibration markers to better align the virtual calibration marker images with the real calibration markers. ### Claim 7 (depends on 6) The method of claim 6, wherein the at least one real calibration marker is tracked by the localizer. ### Claim 8 (depends on 6) The method of claim 6, wherein at least one real object is tracked by the localizer, the at least one real object having a predetermined spatial relationship to the at least one real calibration marker. ### Claim 9 (depends on 6) The method of claim 6, wherein the step of registering is performed in response to the user directing the head-mounted display toward the real calibration markers. ### Claim 10 (depends on 6) The method of claim 6, wherein the step of registering is performed in response to the user directing the head-mounted display toward the real calibration markers so that the real calibration markers are within images captured by a camera comprised in the head-mounted display. ### Claim 11 (depends on 6) The method of claim 6, wherein the step of registering is performed in response to one or more position sensors comprised in the surgical navigation localizer sensing tracking markers comprised in a registration device, wherein the registration device further comprises the real calibration markers. ### Claim 12 (depends on 6) The method of claim 6, wherein receiving input from the user comprises receiving input that alters coordinates of the virtual calibration marker images in the localizer coordinate system. ### Claim 13 (depends on 12) The method of claim 12, wherein receiving input from the user comprises receiving input that alters the coordinates by incremental amounts in any one or more of three directions corresponding to x, y and/or z-axes of the localizer coordinate system. ### Claim 14 (depends on 6) The method of claim 6, wherein the real calibration markers are similar to registration markers used in registering the HMD coordinate system and the localizer coordinate system. ### Claim 15 (depends on 1) The method of claim 1, wherein the at least one real calibration marker is tracked by the localizer and wherein the method is started in response to the user directing a viewing direction of the user toward the real calibration markers, wherein the head-mounted display is configured to be worn by the user. ### Claim 16 (depends on 1) The method of claim 1, wherein at least one real object is tracked by the localizer, the at least one real object having a predetermined spatial relationship to the at least one real calibration marker and wherein the method is started in response to the user directing a viewing direction of the user toward the real calibration markers, wherein the head-mounted display is configured to be worn by the user. ### Claim 17 (depends on 6) The method of claim 6, wherein the at least one real calibration marker is tracked by the localizer and wherein the step of registering is performed in response to the user directing the head-mounted display toward the real calibration markers. ### Claim 18 (depends on 6) The method of claim 6, wherein at least one real object is tracked by the localizer, the at least one real object having a predetermined spatial relationship to the at least one real calibration marker and wherein the step of registering is performed in response to the user directing the head-mounted display toward the real calibration markers.