# US 10,482,614 B2 · Method and system for registration verification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,614 B2 - Document ID: US10482614B2 - Status: Granted - Kind: B2 - Application number: 16/095,078 - Filed: 2017-04-19 - Published: 2019-04-11 - Granted: 2019-11-19 - Earliest claimed: 2016-04-21 - Assignees: Elbit Systems Ltd. - Applicants: Elbit Systems Ltd. - Inventors: Rani Ben-Yishai - Web page: https://patentgrep.com/patent/US10482614B2 ## CPC - A61B 90/37: {Surgical systems with images on a monitor during operation} - A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - G06T 7/337: {involving reference images or patches} - G06T 7/75: {involving models} - G06T 11/60: Creating or editing images; Combining images with text - A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2057: {Details of tracking cameras} - A61B 2034/2063: {Acoustic tracking systems, e.g. using ultrasound} - A61B 2034/2068: {using pointers, e.g. pointers having reference marks for determining coordinates of body points} - A61B 2034/2074: {Interface software} - A61B 2090/363: {Use of fiducial points} - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61B 2090/372: {Details of monitor hardware} - A61B 2090/373: {using light, e.g. by using optical scanners} - A61B 2090/374: {NMR or MRI} - A61B 2090/376: {using X-rays, e.g. fluoroscopy} - A61B 2090/3762: {using computed tomography systems [CT]} - A61B 2090/378: {using ultrasound} - A61B 2090/3945: {Active visible markers, e.g. light emitting diodes} - A61B 2090/395: {with marking agent for marking skin or other tissue} - A61B 2090/3966: {Radiopaque markers visible in an X-ray image} - A61B 2090/502: {Headgear, e.g. helmet, spectacles} - G06T 2207/30204: Marker - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Method for verifying registration of a model of an internal-body-part with the internal-body-part in a reference coordinate system. The internal-body-part is at least partially unseen directly by a user. The method includes the procedures of continuously determining a position and orientation of a head-mounted-display in the reference coordinate system, determining a display location of at least one virtual marker, and displaying the virtual marker according to the display location. The display location of the virtual marker is determined according to the expected position of a respective at least one reference point relative to the head-mounted-display. The reference point is directly visible to the user. The relative position between the reference point and the internal-body-part is substantially constant. The position of the reference point in the reference coordinate system is predetermined. When the model and the internal-body-part are effectively registered, the reference point and corresponding virtual marker appears visually in alignment. ## Claims ### Claim 1 A method for verifying registration of a model of an internal body part with the internal body part in a reference coordinate system, said model and said internal body part previously registered in said reference coordinate system, said internal body part being at least partially unseen directly by a user, the method comprising the procedures of: continuously determining a position and orientation of a head mounted display in said reference coordinate system; determining a display location of at least one virtual marker according to the expected position of a respective at least one reference point relative to said head mounted display, said at least one reference point being directly visible to said user, the position of said at least one reference point in said reference coordinate system is predetermined; displaying said at least one virtual marker according to said display location, wherein, when said model and said internal body part are effectively registered, said at least one reference point and said corresponding at least one virtual marker appears visually in alignment, wherein when said model and said internal body part effectively fall out of registration, said at least one reference point and corresponding virtual marker appear visually out of alignment. ### Claim 2 (depends on 1) The method according to claim 1, further including the procedures of registering said model with said internal body part in said reference coordinate system; and determining the position of said at least one reference point in said reference coordinate system. ### Claim 3 (depends on 2) The method according to claim 2, wherein said position of said at least one reference point is determined relative to said body part by marking said at least one reference point on said model, when said at least one reference point is visible on said model and directly visible by said user. ### Claim 4 (depends on 2) The method according to claim 2, wherein said position of said at least one reference point is determined relative to said body part by determining the position of a tip of tool in said reference coordinate system when said tip touches each of said at least one reference point. ### Claim 5 (depends on 2) The method according to claim 2, wherein said position of said at least one reference point is determined relative to said body part by determining the position and orientation of said head mounted display while said user designates each one of said at least one reference point from at least two view angles respective of at least two positions of said user. ### Claim 6 (depends on 1) The method according to claim 1, wherein said at least one reference point is one of: a marking on the skin; an adhesive marker attached to the skin; a part of a fixture rigidly attached to said body part; and an anatomical feature. ### Claim 7 (depends on 1) The method according to claim 1, wherein said model is at least one of: a one dimensional model; a two dimensional image; a three dimensional segmentation based on one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data; a volume rendering visualization of one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data. ### Claim 8 (depends on 7) The method according to claim 7, wherein said two dimensional image is one of a pre-acquired two dimensional image and an intra-operative two dimensional imager. ### Claim 9 A system for verifying registration of a model of a body part with the internal body part in a reference coordinate system, said model and said internal body part previously registered in said reference coordinate system, said body part being at least partially unseen directly by a user, the system comprising: a head mounted display to be donned by said user; a head mounted display tracking module for continuously determining information relating to the position and orientation of said head mounted display in said reference coordinate system; and a processor, coupled with said head mounted display tracking module, determining the position and orientation of said head mounted display in said reference coordinate system and determining the display location of at least one virtual marker according to the position of a respective at least one reference point relative to said head mounted display, said reference point being visible to said user, the position of said at least one reference point in said reference coordinate system is predetermined, said processor rendering said at least one virtual marker to be displayed on said head mounted display, wherein, when said model and said body part are effectively registered, said at least one reference point and corresponding virtual marker appear on said head mounted display visually in alignment, wherein, when said model and said body part effectively fall out of registration, said at least one reference point and corresponding virtual marker appear on said head mounted display visually out of alignment. ### Claim 10 (depends on 9) The system according to claim 9, wherein said processor registers said model with said reference coordinate system, and wherein said processor determines the position of said at least one reference point. ### Claim 11 (depends on 10) The method according to claim 10, wherein said position of said at least one reference point is determined relative to said body part by marking said at least one reference point on said model, when said at least one reference point is visible on said model and directly visible by said user. ### Claim 12 (depends on 10) The system according to claim 10, wherein said processor determines said position of said of said at least one reference point relative to said body part by determining the position and orientation of said head mounted display while said user designates each one of said at least one reference point from at least two view angles respective of at least two positions of said user. ### Claim 13 (depends on 9) The system according to claim 9, wherein said at least one reference point is one of: a marking on the skin; an adhesive marker attached to the skin; a part of a fixture rigidly attached to said body part; and an anatomical feature. ### Claim 14 (depends on 9) The system according to claim 9, wherein said tracking module includes a head mounted display tracking unit and a reference tracking unit, wherein, said head mounted display tracking unit being fixedly attached to said head mounted display, and wherein said reference tracking unit is fixedly placed relative to said internal body part. ### Claim 15 (depends on 14) The system according to claim 14, wherein said head mounted display tracking unit includes an optical detector and at least one head mounted display light emitter, wherein said reference tracking unit includes a reference optical detector and at least one reference light emitter, wherein the total number of light emitters is at least three, and wherein said reference optical detector acquires at least one image of said at least one head mounted display light emitter and said head mounted display optical detector acquires at least one image of said at least one reference light emitter. ### Claim 16 (depends on 14) The system according to claim 14, wherein said head mounted display tracking unit includes an optical detector, and wherein said reference tracking unit includes at least three reference light emitters. ### Claim 17 (depends on 14) The system according to claim 14, wherein said head mounted display tracking unit includes at least three light emitters, and wherein said reference tracking unit includes an optical detector. ### Claim 18 (depends on 14) The system according to claim 14, wherein said head mounted display tracking unit and said reference tracking unit each includes at least three light emitters, and wherein said tracking module further includes at least one optical detector placed near the surgical field, such that a clear line-of-sight exists between said at least one optical detector and said surgical field. ### Claim 19 (depends on 14) The system according to claim 14, wherein said tracking module further includes a tool tracking unit for determining the position and orientation of a tool. ### Claim 20 (depends on 19) The system according to claim 19, wherein said processor determines said position of said of said at least one reference point relative to said body part by determining the position of a tip of tool in said reference coordinate system when said tip touches each of said at least one reference point. ### Claim 21 (depends on 9) The system according to claim 9, wherein said model is at least one of: a one dimensional model; a two dimensional image; is a three dimensional segmentation based on one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data; a volume rendering of one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data. ### Claim 22 (depends on 9) The system according to claim 9, wherein said two dimensional image is one of a pre-acquired two dimensional image and an intra-operative two dimensional imager. ### Claim 23 (depends on 9) The system according to claim 9, further including a camera coupled with said processor, said camera acquiring an image from the point of view of the user, said image including representations of said at least one reference point and virtual markers, said camera providing the acquired image to said processor, said processor detecting the alignment or misalignment between said at least one reference point and corresponding virtual marker. ### Claim 24 A method for verifying registration of a model of an internal body part with the internal body part in a reference coordinate system, said model and said internal body part previously registered in said reference coordinate system, said internal body part being at least partially unseen directly by a user, the method comprising the procedures of: acquiring an image of at least one reference point from a known position and orientation in said reference coordinate system; identifying the at least one reference point in said acquired image and determining the image location of said at least one reference point, said at least one reference point being directly visible to said user, the position of said at least one reference point in said reference coordinate system is predetermined; determining the expected image location of at least one reference point according to the position of said at least one reference point and said position and orientation from which said image was acquired in the reference coordinate system; determining the congruence between said predicted and determined image location of said at least one reference point, wherein when said model and said internal body part are registered, said predicted and determined image locations are congruent with each other, wherein when said model and said internal body part fall out of registration, said predicted and determined image locations are not congruent with each other. ### Claim 25 (depends on 24) The method according to claim 24, further including the procedures of registering said model with said internal body part in said reference coordinate system; and determining the position of at least one reference point in said reference coordinate system. ### Claim 26 (depends on 24) The method according to claim 24, wherein said image is one of an interrogation image and a tracking image.