# US 2022/0284602 A1 · Systems and methods for enhancement of 3d imagery and navigation via integration of patient motion data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0284602 A1 - Document ID: US20220284602A1 - Status: Published application - Kind: A1 - Application number: 17/649,776 - Filed: 2022-02-02 - Published: 2022-09-08 - Earliest claimed: 2021-03-08 - Assignees: Medtronic Navigation, Inc. - Applicants: Medtronic Navigation, Inc. - Inventors: Patrick A. Helm; Robert J. Pahl - Web page: https://patentgrep.com/patent/US20220284602A1 ## CPC - G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration - G06T 7/33: using feature-based methods - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 90/37: {Surgical systems with images on a monitor during operation} - A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers} - A61B 2034/2048: {using an accelerometer or inertia sensor} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2055: {Optical tracking systems} - A61B 2034/2063: {Acoustic tracking systems, e.g. using ultrasound} - A61B 2090/374: {NMR or MRI} - A61B 2090/3762: {using computed tomography systems [CT]} - G06T 2207/10081: Computed x-ray tomography [CT] - G06T 2207/10088: Magnetic resonance imaging [MRI] - G06T 2207/30004: Biomedical image processing - G06T 2207/30204: Marker - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract Devices, systems, and methods used to register medical image data with anatomical positions are disclosed. The image data is captured by an imaging system over a period of time. The positions are determined by one or more modalities over the period of time. The positions are collected into position data, and determinations are made as to which position data is best for registering the image data with patient anatomy and for reconstructing 3D images of the patient anatomy. ## Claims ### Claim 1 A method for registering image data with anatomy of a patient, the method comprising: capturing, via an imaging system, image data of patient anatomy for a period of time; determining, via a plurality of modalities, position data of the patient anatomy during the period of time that the image data is being captured; and correcting inaccuracies in the image data using the position data. ### Claim 2 (depends on 1) A method according to claim 1, further comprising registering the image data with the patient anatomy based on the position data of the patient anatomy determined by one or more of the plurality of modalities. ### Claim 3 (depends on 1) A method according to claim 1, wherein the plurality of modalities are selected from one or more of an optical tracking system, a three-dimensional camera, radiofrequency modules, ultrasonic transducers, and inertial sensors. ### Claim 4 (depends on 1) A method according to claim 1, further comprising detecting, via the imaging system, locations of a fiducial marker during the period of time, wherein the fiducial marker is coupled to the patient, wherein the locations of the fiducial marker detected during the period of time are incorporated into the position data of the patient anatomy used to register the image data with the patient anatomy. ### Claim 5 (depends on 1) A method according to claim 1, further comprising determining, via the plurality of modalities, position data of the imaging system during the period of time that the image data is being captured, and updating the position data of the patient anatomy based on the position data of the imaging system. ### Claim 6 (depends on 1) A method according to claim 1, further comprising designating tiers to the imaging system and each of the plurality of modalities, such tiers defining a hierarchy of tiers of modalities. ### Claim 7 (depends on 6) A method according to claim 6, wherein the designation of tiers is dynamically updated during or after the period of time that the image data is being captured. ### Claim 8 (depends on 6) A method according to claim 6, further comprising selecting a chief modality from the imaging system and each of the plurality of modalities based on the hierarchy of tiers of modalities, such chief modality providing a coordinate system for registering the image data with the patient anatomy. ### Claim 9 (depends on 1) A method according to claim 1, wherein a first modality is used to observe bulk motion of patient anatomy and a second modality is used to observe deformations of patient anatomy. ### Claim 10 (depends on 1) A method according to claim 1, wherein selection of which of the plurality of modalities to use for registering the image data with the patient anatomy is based on validation criteria for establishing accuracy of each of the plurality of modalities. ### Claim 11 A system comprising: a first modality configured to capture a first set of position data of patient anatomy; a second modality configured to capture a second set of position data of patient anatomy; and a processor configured to: receive image data of a patient; receive the first set of position data of patient anatomy and the second set of position data of patient anatomy; and correct inaccuracies in the image data using the first set of position data of patient anatomy and the second set of position data of patient anatomy. ### Claim 12 (depends on 11) A system according to claim 11, wherein the processor is further configured to register the image data with patient anatomy based on at least one of the first set of position data of patient anatomy and the second set of position data of patient anatomy. ### Claim 13 (depends on 11) A system according to claim 11, wherein the plurality of modalities are selected from one or more of an optical tracking system, a three-dimensional camera, radiofrequency modules, ultrasonic transducers, and inertial sensors. ### Claim 14 (depends on 11) A system according to claim 11, wherein the processor is further configured to designate tiers to the first modality and the second modality, such tiers defining a hierarchy of tiers of modalities. ### Claim 15 (depends on 14) A system according to claim 14, further comprising selecting a chief modality from the first modality and the second modality based on the hierarchy of tiers of modalities, such chief modality providing a coordinate system for registering the image data with the patient anatomy. ### Claim 16 (depends on 11) A system according to claim 11, wherein the processor is configured to select which of the first set of position data of patient anatomy and the second set of position data of patient anatomy to use for registration of the image data with the patient anatomy based on validation criteria for establishing accuracy of each of the first modality and the second modality. ### Claim 17 (depends on 11) A system according to claim 11, wherein the processor is further configured to reconstruct a 3D image of the patient anatomy from the image data and at least one of the first set of position data of patient anatomy and the second set of position data of patient anatomy. ### Claim 18 A computing apparatus comprising: a processor; and memory storing instructions that, when executed by the processor, configure the apparatus to: capture, via an imaging system, image data of patient anatomy for a period of time; determine, via a plurality of modalities, position data of the patient anatomy during the period of time that the image data is being captured; and register the image data with the patient anatomy based on the position data of the patient anatomy. ### Claim 19 (depends on 18) A computing apparatus according to claim 18, wherein the plurality of modalities are selected from one or more of an optical tracking system, a three-dimensional camera, radiofrequency modules, ultrasonic transducers, and inertial sensors ### Claim 20 (depends on 18) A computing apparatus according to claim 18, wherein the apparatus is further configured to designate tiers to the imaging system and each of the plurality of modalities, such tiers defining a hierarchy of tiers of modalities.