# US 2024/0202927 A1 · Method for generating a 3d physical model of a patient specific anatomic feature from 2d medical images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202927 A1
- Document ID: US20240202927A1
- Status: Published application
- Kind: A1
- Application number: 18/595,213
- Filed: 2024-03-04
- Published: 2024-06-20
- Earliest claimed: 2016-10-14
- Assignees: Axial Medical Printing Limited
- Applicants: Axial Medical Printing Limited
- Inventors: Niall HASLAM; Lorenzo TROJAN; Daniel CRAWFORD
- Web page: https://patentgrep.com/patent/US20240202927A1

## CPC

- G06T 7/11: Region-based segmentation
- B33Y 50/00: Data acquisition or data processing for additive manufacturing
- G06F 18/24: Classification techniques
- G06T 7/0014: {using an image reference approach}
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/50: for simulation or modelling of medical disorders
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2207/30004: Biomedical image processing
- G06T 2210/41: Medical
- G06T 2219/028: Multiple view windows (top-side-front-sagittal-orthogonal)
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B33Y: ADDITIVE MANUFACTURING, i.e. MANUFACTURING OF THREE-DIMENSIONAL [3D] OBJECTS BY ADDITIVE DEPOSITION, ADDITIVE AGGLOMERATION OR ADDITIVE LAYERING, e.g. BY 3D PRINTING, STEREOLITHOGRAPHY OR SELECTIVE LASER SINTERING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

There is provided a method for generating a 3D physical model of a patient specific anatomic feature from 2D medical images. The 2D medical images are uploaded by an end-user via a Web Application and sent to a server. The server processes the 2D medical images and automatically generates a 3D printable model of a patient specific anatomic feature from the 2D medical images using a segmentation technique. The 3D printable model is 3D printed as a 3D physical model such that it represents a 1:1 scale of the patient specific anatomic feature. The method includes the step of automatically identifying the patient specific anatomic feature.

## Claims

### Claim 1

A computer-implemented method for defining patient specific anatomical features from medical images, the computer-implemented method comprising:
receiving medical images with information on bone and/or soft tissue of a patient;
applying the information as an input to a segmentation algorithm to automatically process the medical images to assign a label for each pixel of the medical images based at least partially on the bone and/or soft tissue;
accessing a database of medical image anatomical features;
using an anatomical feature identification algorithm to probabilistically match the labeled pixels of the medical images against the database of medical image anatomical features to generate segmentation data that defines one or more patient specific anatomical features based on the labeled pixels of the medical images; and
generating a 3D model of the one or more patient specific anatomical features comprising the bone and/or soft tissue using the generated segmentation data.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the information on bone and/or soft tissue of the patient comprises information of a type of the bone and/or soft tissue within the medical images.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the information of the type of bone and/or soft tissue is inputted by a user.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein the information of the type of bone and/or soft tissue is captured from the medical images.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the segmentation algorithm comprises at least one of the following multiple segmentation techniques: threshold-based, decision tree, chained decision forest, or neural network method.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising combining results of each of the multiple segmentation techniques together to derive a final segmentation result.

### Claim 7 (depends on 5)

The computer-implemented method of claim 5, wherein the segmentation algorithm includes a thresholding algorithm that automatically estimates a threshold value for a pixel being classified as bone or soft tissue, the threshold value optimized based on one or more of the following parameters: scan type, bone type, tissue type, age, gender, and weight of the patient.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the threshold value is estimated from a histogram analysis of the medical images.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the histogram analysis comprises detecting peaks of a 2D medical images histogram corresponding to bone and/or soft tissue similar to bone and/or soft tissue within the medical images and, for each detected peak of the 2D medical images histogram:
deriving or determining a detected peak inflection point by calculating a zero of a 2D medical images histogram second derivative in proximity of a peak; and
deriving or determining an inflection offset between the peak and the inflection point,
wherein the estimated threshold value corresponds to the position of the detected peak with an offset corresponding to three times the derived or determined inflection offset.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein, if an error occurs during peak detection of the 2D medical images histogram, the thresholding algorithm applies a default threshold value based on at least one of the bone and/or soft tissue of the patient or a scan type of the medical images.

### Claim 11 (depends on 7)

The computer-implemented method of claim 7, wherein the segmentation algorithm includes the thresholding algorithm in combination with a smoothing algorithm, the smoothing algorithm comprising a Gaussian filter.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein the thresholding algorithm uses a logistic probabilistic function to calculate a likelihood of a pixel being a bone or soft tissue in question.

### Claim 13 (depends on 7)

The computer-implemented method of claim 7, wherein the thresholding algorithm is configured to segment out bone from background information within the medical images.

### Claim 14 (depends on 7)

The computer-implemented method of claim 7, wherein the bone type is selected from a list consisting of hard bone and soft bone.

### Claim 15 (depends on 7)

The computer-implemented method of claim 7, wherein the scan type is selected from a list consisting of CT, MRI, PET and SPECT.

### Claim 16 (depends on 7)

The computer-implemented method of claim 7, wherein the scan type is inputted by a user.

### Claim 17 (depends on 7)

The computer-implemented method of claim 7, wherein the segmentation algorithm includes a chained decision forest, in which a hierarchy of decision forests is used, wherein results of a decision tree and results from the thresholding algorithm are fed to a new decision tree alongside pixel values of the medical images.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, wherein each forest-node of the chained decision forest is treated as a simple classifier that produces a score indicative of a likelihood of a pixel being a bone or soft tissue in question.

### Claim 19 (depends on 17)

The computer-implemented method of claim 17, wherein the decision tree is created based on one or more of the following properties of medical images pixels: number of pixels resembling bone or soft tissue of interest located near a pixel in question, number of pixels resembling bone or soft tissue of interest located near a given pixel, or overall gradient of the image at a given pixel.

### Claim 20 (depends on 17)

The computer-implemented method of claim 17, wherein the decision tree is applied to a subset of pixels within the medical images and labels obtained from the subset are then up scaled using standard interpolation methods to recover segmentation of the full image and, wherein the subset of pixels is generated by subsampling the medical images, and wherein a subsampling stride is selected depending on pixel size.

### Claim 21 (depends on 5)

The computer-implemented method of claim 5, wherein the segmentation algorithm includes a Neural Network method comprising at least one of a Fully Convolutional Neural Network (FCNN) or a UNET Neural network, and wherein the Neural Network is trained from a database of existing medical images that have been labelled and a medical imaging ontology.

### Claim 22 (depends on 1)

The computer-implemented method of claim 1, wherein the database of medical image anatomical features comprises a graph database comprising nodes representing anatomic features including tissue type or organ type, and edges associated with relationships between the nodes, including: has part, proximity, attachment, ligament, or functional, wherein a node includes: a reference to a medical image with a corresponding anatomic feature, a reference to results of segmentation of a medical image with a corresponding anatomic feature, information relating to the anatomic feature including volume, surface area, or Hounsfield Unit standard deviation or average, and wherein the graph database is updated after the generation of a 3D model.

### Claim 23 (depends on 1)

The computer-implemented method of claim 1, further comprising:
detecting touching bones or soft tissues within the 2D medical images; and
using an edge finding algorithm to separate different bones or soft tissues.

### Claim 24 (depends on 1)

The computer-implemented method of claim 1, further comprising detecting a pathology of the one or more patient specific anatomical features comprising the bone and/or soft tissue by measuring deviation of the 3D model from an expected appearance of the one or more patient specific anatomical features comprising the bone and/or soft tissue.

### Claim 25 (depends on 24)

The computer-implemented method of claim 24, wherein the pathology of the one or more patient specific anatomical features comprising the bone and/or soft tissue comprises a skeletal deformity.

### Claim 26 (depends on 1)

The computer-implemented method of claim 1, wherein the medical images are MRI images.

### Claim 27

A system for defining patient specific anatomical features from medical images, the system comprising at least one processor configured to:
receive medical images with information on bone and/or soft tissue of a patient;
apply the information as an input to a segmentation algorithm to automatically process the medical images to assign a label for each pixel of the medical images based at least partially on the bone and/or soft tissue;
access a database of medical image anatomical features;
use an anatomical feature identification algorithm to probabilistically match the labeled pixels of the medical images against the database of medical image anatomical features to generate segmentation data that defines one or more patient specific anatomical features based on the labeled pixels of the medical images; and
generate a 3D model of the one or more patient specific anatomical features comprising the bone and/or soft tissue using the generated segmentation data.

### Claim 28 (depends on 27)

The system of claim 27, wherein the information on bone and/or soft tissue of the patient comprises information of a type of the bone and/or soft tissue within the medical images.

### Claim 29 (depends on 27)

The system of claim 27, wherein the segmentation algorithm comprises at least one of the following multiple segmentation techniques: threshold-based, decision tree, chained decision forest, or neural network method.

### Claim 30 (depends on 29)

The system of claim 29, wherein the segmentation algorithm includes a thresholding algorithm that automatically estimates a threshold value for a pixel being classified as bone or soft tissue, the threshold value optimized based on one or more of the following parameters: scan type, bone type, tissue type, age, gender, and weight of the patient.
