# US 12,118,727 B2 · System and method for training a machine learning model and for providing an estimated interior image of a patient > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,727 B2 - Document ID: US12118727B2 - Status: Granted - Kind: B2 - Application number: 17/596,287 - Filed: 2020-06-01 - Published: 2022-07-21 - Granted: 2024-10-15 - Earliest claimed: 2019-06-13 - Assignees: Raysearch Laboratories AB (publ) - Applicants: RaySearch Laboratories AB - Inventors: Sebastian Andersson; Kjell Eriksson; Stina Svensson; Ola Weistrand - Web page: https://patentgrep.com/patent/US12118727B2 ## CPC - G06T 7/0016: {involving temporal comparison} - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - G06T 7/33: using feature-based methods - G16H 30/40: for processing medical images, e.g. editing - G06T 2207/10076: 4D tomography; Time-sequential 3D tomography - G06T 2207/10081: Computed x-ray tomography [CT] - G06T 2207/10088: Magnetic resonance imaging [MRI] - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30004: Biomedical image processing - G06T 2207/30096: Tumor; Lesion - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A deep learning model may be trained to provide an estimated image of the interior of a patient, based on a number of image sets, each image set comprising an interior image of the interior of a person and a contour image of the person's outer contour at a specific point in time. The model is trained to establish an optimized parametrized conversion function G specifying the correlation between the interior of the person and the persons outer contour based on the image sets. The conversion function G can then be used to provide estimated images of patient's interior based on their contours. ## Claims ### Claim 1 A computer-based method of training a deep learning model for providing an estimated image of an interior of a patient, based on a number of image sets, each image set comprising a first interior image of an interior of a person and a contour image of the person's outer contour at a specific point in time, and a second interior image of the interior of the person, comprising the steps of: a. submitting the image sets to a deep learning model; and b. training the model to establish an optimized parametrized conversion function G specifying the correlation between the interior of the person and the person's outer contour by, for each image set, applying the model to the contour image and the second interior image of the image set, comparing the output to the first interior image of the image set and using the result of the comparison to train the model. ### Claim 2 (depends on 1) A computer-based method according to claim 1, wherein the training step comprises: obtaining an initial parametrized conversion function G arranged to convert a contour image into an estimated image of the interior of the patient based on a first image set of the number of image sets; obtaining a first estimated interior image by applying the parametrized conversion function G to the contour image and the second interior image of the first image set; comparing, in a first comparing step, the first estimated interior image to the interior image of the first image set; and adjusting the initial parametrized conversion function G based on the comparison, to form a first parametrized conversion function G1, which is applied to the second image set. ### Claim 3 (depends on 2) A computer-based method according to claim 2, wherein the training step further comprises: applying the first parametrized conversion function G1 to the contour image of the second image set to obtain a second estimated image; comparing the second estimated image to the interior image of the second image pair, and adjust the initial parametrized conversion function G to form a second parametrized conversion function G2. ### Claim 4 (depends on 2) A computer-based method according to claim 2, wherein the training step further comprises: obtaining a second estimated interior image by applying the parametrized conversion function G to the contour image of the second image set; comparing, in a second comparing step, the second estimated interior image to the interior image of the second image pair; and adjusting the initial parametrized conversion function G based on the first and the second comparing step, to form a first parametrized conversion function G1. ### Claim 5 (depends on 1) A method according to claim 1, wherein the first and second interior images of each image set are segmentation maps and the model is trained to output a segmentation map. ### Claim 6 (depends on 1) A method according to claim 1, wherein the first second interior images of each image set are CT images and the model is trained to output a synthetic CT image. ### Claim 7 (depends on 1) A method according to claim 1, wherein each image set further comprises at least one slice of an MR image to provide additional information about the patient's interior. ### Claim 8 (depends on 1) A method according to claim 1, wherein the first interior image and the contour image of each image set are 4D images and the model is trained to output a synthetic 4D image. ### Claim 9 A computer-based method of providing an estimated image of the an interior of a patient at a first point in time, comprising: a. providing an interior image of the interior of the patient at a second point in time, preceding the first point in time, to a deep learning model comprising an optimized parametrized conversion function based on the correlations between contours and interiors of persons; b. providing a contour image of the contour of the patient at the first point in time to the deep learning model; and c. outputting from the deep learning model an estimated image of the patient based on the interior image, the contour image and optimized parametrized conversion function G. ### Claim 10 (depends on 9) A method according to claim 9, wherein the contour image is based on data obtained from a surface scanning device. ### Claim 11 (depends on 9) A method according to claim 9, wherein the estimated image is a segmented image of the patient. ### Claim 12 (depends on 9) A method according to claim 9, wherein the estimated image is a CT image of the patient. ### Claim 13 (depends on 9) A method according to claim 9, wherein the steps b and c are repeated for several subsequent contour images to produce a set of estimated images constituting a 4D image. ### Claim 14 A computer program product stored on a non-transitory medium which, when executed in a processor in a computer, is arranged to cause the computer to perform a method comprising: a. providing an interior image of an interior of the a patient at a second point in time, preceding a first point in time, to a deep learning model comprising an optimized parametrized conversion function based on the correlations between contours and interiors of persons; b. providing a contour image of the contour of the patient at the first point in time to the deep learning model; and c. outputting from the deep learning model an estimated image of the patient based on the interior image, the contour image and optimized parametrized conversion function G. ### Claim 15 (depends on 14) A computer system comprising a processor, and a program memory, said program memory comprising a computer program product according to claim 14.