# US 11,304,623 B2 · Integration of medical imaging and location tracking > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,304,623 B2 - Document ID: US11304623B2 - Status: Granted - Kind: B2 - Application number: 16/231,990 - Filed: 2018-12-25 - Published: 2020-06-25 - Granted: 2022-04-19 - Earliest claimed: 2018-12-25 - Assignees: Biosense Webster (Israel) Ltd. - Applicants: Biosense Webster (Israel) Ltd. - Inventors: Andres Claudio Altmann; Vadim Gliner; Assaf Govari - Web page: https://patentgrep.com/patent/US11304623B2 ## CPC - A61B 5/062: {using magnetic field} - G06T 7/337: {involving reference images or patches} - A61B 5/055: involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging - A61B 6/032: {Transmission computed tomography [CT]} - A61B 6/466: {adapted to display 3D data} - A61B 6/584: {determining position of components of the apparatus or device using images of the phantom} - G01R 33/58: Calibration of imaging systems, e.g. using test probes {, Phantoms; Calibration objects or fiducial markers such as active or passive RF coils surrounding an MR active material} - G06T 3/60: Rotation of whole images or parts thereof - G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215) - G06T 7/37: using transform domain methods - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 2207/10081: Computed x-ray tomography [CT] - G06T 2207/10088: Magnetic resonance imaging [MRI] - G06T 2207/30021: Catheter; Guide wire - G06T 2207/30048: Heart; Cardiac - G06T 2207/30204: Marker - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12) ## Abstract A location tracking system maps anatomical structures in a first coordinate system, in a fixed position within a medical imaging system, which captures 3D images in a second coordinate system. The 3D images are converted to and stored in a standardized format in a third coordinate system in accordance with a first coordinate transformation. A first 3D image captured by the imaging system is registered with the first coordinate system so as to produce a second coordinate transformation. The first and second coordinate transformations are combined so as to derive a third coordinate transformation between the first and third coordinate systems. A second 3D image of a body of a subject, captured by the imaging system, is processed in order to extract image features in the third coordinate system. The extracted image features are joined with location data captured by the location tracking system by applying the third coordinate transformation. ## Claims ### Claim 1 A method for registering images, the method comprising: installing a location tracking system, which is configured to map anatomical structures in a first coordinate system, in a fixed position within a medical imaging system, which captures three-dimensional (3D) images of the anatomical structures in a second coordinate system, wherein the three-dimensional (3D) images are converted to and stored in a standardized format in a third coordinate system in accordance with a first coordinate transformation between the second coordinate system and the third coordinate system; registering a first 3D image captured by the imaging system with the first coordinate system so as to produce a second coordinate transformation between the first coordinate system and the second coordinate system; combining the first coordinate transformation and the second coordinate transformation so as to derive a third coordinate transformation between the first coordinate system and the third coordinate system; processing a second 3D image of a body of a subject captured by the imaging system in order to extract image features in the third coordinate system; and joining the extracted image features with location data captured by the location tracking system by applying the third coordinate transformation, further comprising the steps of: a registration step, wherein an MRI scanner and the location tracking system are registered with each other, a second coordinate transformation step, wherein a processor calculates the second coordinate transformation between the second coordinate system and the first coordinate system, wherein the second coordinate system is a device coordinate system and the first coordinate system is a location coordinate system, a patient insertion step, wherein the body of the subject is inserted into the MRI scanner, a volume of interest step, wherein a low-resolution scan of the body of the subject is performed using the MRI scanner, and wherein an operator defines, using a control console, a volume of interest as a volume for subsequent high-resolution MRI scans, a first coordinate transformation step, wherein the first coordinate transformation is calculated between the second coordinate system and the third coordinate system, wherein the third coordinate system is a patient coordinate system, a third coordinate transformation step, wherein the third coordinate transformation is calculated between the first coordinate system and the third coordinate system as a product of the first coordinate transformation and the second coordinate transformation, a start step, wherein imaging of the body of the subject and location tracking are started, an MRI scan step wherein, the volume of interest is scanned by the MRI scanner with a high-resolution scan, an image generation step, comprising generating a medical image in the third coordinate system, a read step, wherein the medical image is read by the processor, in a processing step in which the medical image is further processed to produce a processed image, a tracking step, wherein location tracking of the catheter by the location tracking system is started, a location data step, wherein the location tracking produces location data in the first coordinate system, a transformation application step, wherein the third coordinate transformation is applied to the location data from the location data step, wherein the location data is transformed into Digital Imaging and Communications in Medicine (DICOM) protocol coordinates, and a combination step, wherein the processed image from the processing step and the location data in the Digital Imaging and Communications in Medicine (DICOM) protocol coordinates from the transformation application step are combined to create a joint image. ### Claim 2 (depends on 1) The method according to claim 1, wherein the location tracking system comprises a magnetic tracking system. ### Claim 3 (depends on 2) The method according to claim 2, wherein registering the first 3D image with the first coordinate system comprises inserting a jig comprising calibration targets into the medical imaging system, capturing the jig in the first 3D image, and measuring locations of the calibration targets. ### Claim 4 (depends on 1) The method according to claim 1, wherein the medical imaging system comprises a magnetic resonance imaging (MRI) system. ### Claim 5 (depends on 1) The method according to claim 1, wherein the medical imaging system comprises a computerized tomography (CT) system. ### Claim 6 (depends on 1) The method according to claim 1, wherein the third coordinate system is defined according to the Digital Imaging and Communications in Medicine (DICOM) protocol. ### Claim 7 (depends on 6) The method according to claim 6, wherein processing the second 3D image comprises reading and processing the second 3D image by a software application complying with the DICOM protocol. ### Claim 8 (depends on 1) The method according to claim 1, wherein processing the second 3D image comprises at least one of rotating and segmenting the image. ### Claim 9 (depends on 1) The method according to claim 1, wherein the location data captured by the location tracking system comprises a location of a distal end of a catheter within an anatomical structure in the body. ### Claim 10 (depends on 1) The method according to claim 1, wherein joining the extracted image features with location data comprises displaying the extracted image features and the location data concurrently on a display. ### Claim 11 An apparatus for displaying registered images, the apparatus comprising: a location tracking system, which is configured to map anatomical structures in a first coordinate system; a medical imaging system, within which the location tracking system is installed in a fixed position, and which is configured to capture three-dimensional (3D) images of the anatomical structures in a second coordinate system, wherein the three-dimensional (3D) images are converted to and stored in a standardized format in a third coordinate system in accordance with a first coordinate transformation between the second coordinate system and the third coordinate system; a processor configured to: register a first 3D image captured by the imaging system with the first coordinate system so as to produce a second coordinate transformation between the first coordinate system and the second coordinate system; combine the first coordinate transformation and the second coordinate transformation so as to derive a third coordinate transformation between the first coordinate system and the third coordinate system; process a second 3D image of a body of a subject captured by the imaging system in order to extract image features in the third coordinate system; and join the extracted image features with location data captured by the location tracking system after applying the third coordinate transformation to the location data, wherein the apparatus is configured to be employed in performance of the following steps: a registration step, wherein an MRI scanner and the location tracking system are registered with each other, a second coordinate transformation step, wherein a processor calculates the second coordinate transformation between the second coordinate system and the first coordinate system, wherein the second coordinate system is a device coordinate system and the first coordinate system is a location coordinate system, a patient insertion step, wherein the body of the subject is inserted into the MRI scanner, a volume of interest step, wherein a low-resolution scan of the body of the subject is performed using the MRI scanner, and wherein an operator defines, using a control console, a volume of interest as a volume for subsequent high-resolution MRI scans, a first coordinate transformation step, wherein the first coordinate transformation is calculated between the second coordinate system and the third coordinate system, wherein the third coordinate system is a patient coordinate system, a third coordinate transformation step, wherein the third coordinate transformation is calculated between the first coordinate system and the third coordinate system as a product of the first coordinate transformation and the second coordinate transformation, a start step, wherein imaging of the body of the subject and location tracking are started, an MRI scan step wherein, the volume of interest is scanned by the MRI scanner with a high-resolution scan, an image generation step, comprising generating a medical image in the third coordinate system, a read step, wherein the medical image is read by the processor, in a processing step in which the medical image is further processed to produce a processed image, a tracking step, wherein location tracking of the catheter by the location tracking system is started, a location data step, wherein the location tracking produces location data in the first coordinate system, a transformation application step, wherein the third coordinate transformation is applied to the location data from the location data step, wherein the location data is transformed into Digital Imaging and Communications in Medicine (DICOM) protocol coordinates, and a combination step, wherein the processed image from the processing step and the location data in the Digital Imaging and Communications in Medicine (DICOM) protocol coordinates from the transformation application step are combined to create a joint image. ### Claim 12 (depends on 11) The apparatus according to claim 11, wherein the location tracking system comprises a magnetic tracking system. ### Claim 13 (depends on 12) The apparatus according to claim 12, wherein the first 3D image is registered with the first coordinate system by inserting a jig comprising calibration targets into the medical imaging system, capturing the jig in the first 3D image, and measuring locations of the calibration targets. ### Claim 14 (depends on 13) The apparatus according to claim 13, wherein the third coordinate system is defined according to the Digital Imaging and Communications in Medicine (DICOM) protocol. ### Claim 15 (depends on 14) The apparatus according to claim 14, wherein the processor is configured to read and process the second 3D image using a software complying with the DICOM protocol. ### Claim 16 (depends on 11) The apparatus according to claim 11, wherein the medical imaging system comprises a magnetic resonance imaging (MRI) system. ### Claim 17 (depends on 11) The apparatus according to claim 11, wherein the medical imaging system comprises a computerized tomography (CT) system. ### Claim 18 (depends on 11) The apparatus according to claim 11, wherein processing the second 3D image comprises at least one of rotating and segmenting of the image. ### Claim 19 (depends on 11) The apparatus according to claim 11, wherein the location data captured by the location tracking system comprises the location of a distal end of a catheter within an anatomical structure in the body. ### Claim 20 (depends on 11) The apparatus according to claim 11, wherein the processor is configured to display the extracted image features concurrently with the location data on a display.