# US 12,008,759 B2 · Method and system for identifying pathological changes in follow-up medical images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,759 B2 - Document ID: US12008759B2 - Status: Granted - Kind: B2 - Application number: 17/101,069 - Filed: 2020-11-23 - Published: 2021-06-03 - Granted: 2024-06-11 - Earliest claimed: 2019-11-29 - Assignees: Siemens Healthineers AG - Applicants: Siemens Healthcare GmbH - Inventors: Gerardo Hermosillo Valadez; Sven Kohle; Christian Tietjen; Matthias Wolf - Web page: https://patentgrep.com/patent/US12008759B2 ## CPC - G06T 7/0016: {involving temporal comparison} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06T 7/38: Registration of image sequences - G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS - G06T 2207/10088: Magnetic resonance imaging [MRI] - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/20224: Image subtraction - G06T 2207/30061: Lung - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A computer-implemented method for identifying pathological changes in follow-up medical images is provided. In an embodiment, the method includes: providing reference image data showing a body part of a patient at a first time; providing follow-up image data showing a body part of a patient at a subsequent second time; generating one or more deformation fields for the reference image data and the follow-up image data describing anatomical deformations in the body part between the reference image data and the follow-up image data using at least one image registration; aligning the reference image data and the follow up image data using the one or more deformation fields to generate co-aligned image data; analyzing the co-aligned image data to identify pathological changes in the body part from the reference image data to the follow image data using a machine learned network trained to recognize pathological relevant changes in co-aligned image data. ## Claims ### Claim 1 A computer-implemented method for identifying pathological changes in follow-up medical images, the computer-implemented method comprising: providing reference image data showing a body part of a patient at a first time; providing follow-up image data showing the body part of the patient at a second time, the second time being after the first time; generating one or more first deformation fields for the reference image data and the follow-up image data using at least one image registration, the one or more first deformation fields describing anatomical deformations in the body part between the reference image data and the follow-up image data; aligning the reference image data and the follow-up image data using the one or more first deformation fields to generate co-aligned image data; and analyzing the co-aligned image data to identify pathological changes using a machine learned network, the pathological changes occurring in the body part of the patient between the first time and the second time, and the machine learned network being trained to identify pathological changes in the co-aligned image data. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the aligning includes calculating deformed reference image data by applying the one or more first deformation fields to the reference image data. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, further comprising: subtracting the deformed reference image data from the follow-up image data; and multiplying a result of the subtracting with an image filter. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the analyzing includes generating an image filter based on the machine learned network, the image filter corresponding to the co-aligned image data, and the image filter being configured to highlight the pathological changes in the body part. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: defining a plurality of corresponding image segments in the reference image data and the follow-up image data, wherein the generating of the one or more first deformation fields includes generating a separate deformation field for each image segment among the plurality of corresponding image segments by applying a separate image registration for each image segment among the plurality of corresponding image segments, each separate deformation field being among the one or more first deformation fields, and each separate image registration being among the at least one image registration. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising: rendering an image of the follow-up image data; and displaying the image, the image highlighting the pathological changes in the body part. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, further comprising: determining an imaging window for at least one of the reference image data or the follow-up image data; and at least one of rendering or displaying only parts of the image within the imaging window. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the at least one image registration includes one or more non-rigid image registrations. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, further comprising: fitting the one or more first deformation fields to one or more motion models for soft tissue deformations. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the follow-up image data and the reference image data are magnetic resonance image data; the reference image data is acquired using a first magnetic resonance pulse sequence; and the follow-up image data is acquired using a second magnetic resonance pulse sequence, the second magnetic resonance pulse sequence being different from the first magnetic resonance pulse sequence. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, wherein the second magnetic resonance pulse sequence is a double inversion recovery sequence or a post-contrast sequence. ### Claim 12 (depends on 1) The computer-implemented method of claim 1, wherein the machine learned network is trained using a loss function derived from one or more second deformation fields generated by a biomechanical model of soft tissue deformation. ### Claim 13 (depends on 1) A non-transitory computer program product storing program elements which cause one or more processors of a system for identifying pathological changes in follow-up medical images, to perform the computer-implemented method of claim 1, when the program elements are executed by the one or more processors. ### Claim 14 (depends on 1) A non-transitory computer-readable medium storing program elements, readable and executable by one or more processors of a system for identifying pathological changes in follow-up medical images, to perform the computer-implemented method of claim 1, when the program elements are executed by the one or more processors. ### Claim 15 A method for training a machine learned network including a neural network to identify pathological changes in follow-up medical images, the method comprising: providing training reference image data showing a body part of a patient; generating simulated follow-up image data by deforming the training reference image data using one or more models for biomechanical soft tissue deformation; calculating one or more deformation fields for the training reference image data and the simulated follow-up image data using at least one image registration, the one or more deformation fields describing anatomical deformations in the body part between the training reference image data and the simulated follow-up image data; aligning the training reference image data and the simulated follow-up image data using the one or more deformation fields to generate co-aligned image data; analyzing the co-aligned image data to identify pathological changes using the machine learned network, the pathological changes occurring in the body part from the training reference image data to the simulated follow-up image data; comparing a result of the analyzing with a known input based on the one or more models for biomechanical soft tissue deformation; and adjusting weights in the neural network based on the comparing. ### Claim 16 (depends on 15) The method of claim 15, wherein the simulated follow-up image data includes one or more simulated pathological changes. ### Claim 17 (depends on 15) A non-transitory computer program product storing program elements which cause one or more processors of a system for identifying pathological changes in follow-up medical images, to perform the method of claim 15, when the program elements are executed by the one or more processors. ### Claim 18 (depends on 15) A non-transitory computer-readable medium storing program elements, readable and executable by one or more processors of a system for identifying pathological changes in follow-up medical images, to perform the method of claim 15, when the program elements are executed by the one or more processors. ### Claim 19 A system for identifying pathological changes in follow-up medical images, the system comprising: an interface unit configured to receive reference image data showing a body part of a patient at a first time, and to receive follow-up image data showing the body part of the patient at a second time, the second time being after the first time; and at least one processor configured to execute computer-readable instructions to generate one or more deformation fields for the reference image data and the follow-up image data using at least one image registration, the one or more deformation fields describing anatomical deformations in the body part between the reference image data and the follow-up image data, align the reference image data and the follow-up image data using the one or more deformation fields to generate co-aligned image data, and analyze the co-aligned image data to identify pathological changes using a machine learned network, the pathological changes occurring in the body part of the patient between the first time and the second time, and the machine learned network being trained to identify pathological changes in the co-aligned image data. ### Claim 20 (depends on 19) The system of claim 19, wherein the at least one processor is configured to execute computer-readable instructions to: align the reference image data and the follow-up image data including calculating deformed reference image data by applying the one or more deformation fields to the reference image data; and analyze the co-aligned image data by generating an image filter based on the machine learned network, the image filter corresponding to the co-aligned image data, and the image filter being configured to highlight the pathological changes in the body part. ### Claim 21 (depends on 20) The system of claim 20, wherein the at least one processor is configured to execute computer-readable instructions to: subtract the deformed reference image data from the follow-up image data; and multiply a result of the subtraction with the image filter.