# US 10,373,328 B2 · Generation of a digital 3D model of an object based on predicting a shape of the object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,373,328 B2
- Document ID: US10373328B2
- Status: Granted
- Kind: B2
- Application number: 16/028,303
- Filed: 2018-07-05
- Published: 2018-11-01
- Granted: 2019-08-06
- Earliest claimed: 2014-01-27
- Assignees: Materialise N.V.
- Applicants: Materialise NV
- Inventors: Jan Maes; Tom Cluckers; Joris Van Deun
- Web page: https://patentgrep.com/patent/US10373328B2

## CPC

- G16H 30/40: for processing medical images, e.g. editing
- G06F 16/58: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F 16/5854: {using shape and object relationship}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 20/00: Machine learning
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/50: Depth or shape recovery
- G06T 7/55: from multiple images
- G06T 7/586: from multiple light sources, e.g. photometric stereo
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/50: for simulation or modelling of medical disorders
- G16Z 99/00: Subject matter not provided for in other main groups of this subclass
- H04N 21/61: Network physical structure; Signal processing (H04B takes precedence)
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/10116: X-ray image
- G06T 2207/30008: Bone
- G06T 2210/41: Medical
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16Z: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS, NOT OTHERWISE PROVIDED FOR
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods for predicting shape are provided. A system for predicting shape can include a database, a training analysis module, a subject analysis module, and a prediction module. The database can store two sets of training models characterized by first and second parameters, respectively (e.g., bone and cartilage), as well as a subject model characterized by the first parameter (e.g., a bone model). The relationships between these models can be determined by a training analysis module and a subject module. Based on these relationships, the prediction module can generate a predicted shape characterized by the second parameter (e.g., a predicted cartilage model corresponding to the bone model).

## Claims

### Claim 1

A system for generating a digital 3D model of an object of an object type based on predicting a shape of the object, the system comprising:
a memory configured to store at least one database configured to store a first set of training models corresponding to 2D models of the object type, a second set of training models corresponding to 3D models of the object type, and a subject model corresponding to a 2D model corresponding to the object, wherein each 2D model in the first set of training models corresponds to a 3D model in the second set of training models;
a processor configured to:
determine an average shape for the first set of training models;
determine a decomposition of the first set of training models;
determine an average shape for the second set of training models;
determine a decomposition of the second set of training models;
determine a relationship between the first set of training models and the second set of training models as function of the average shape for the first set of training models, the decomposition of the first set of training models, the average shape for the second set of training models, and the decomposition of the second set of training models;
determine a relationship between the subject model and the first set of training models; and
generate the digital 3D model of the object based on a predicted shape of the object based on the relationship between the first set of training models and the second set of training models and on the relationship between the subject model and the first set of training models.

### Claim 2 (depends on 1)

The system of claim 1, wherein to determine the relationship between the subject model and the first set of training models comprises to determine a fitting vector.

### Claim 3 (depends on 2)

The system of claim 2, wherein to generate the digital 3D model of the object based on the predicted shape comprises to generate the predicted shape based at least in part on the fitting vector.

### Claim 4 (depends on 1)

The system claim 1, wherein to generate the digital 3D model of the object based on the predicted shape comprises to generate the predicted shape by modifying the subject model.

### Claim 5 (depends on 1)

The system of claim 1, wherein the object type comprises anatomy of a patient.

### Claim 6

A computer implemented method of generating a digital 3D model of an object of an object type based on predicting a shape of the object, the method comprising:
determining an average shape for a first set of training models comprising 2D models of the object type;
determining a decomposition of the first set of training models;
determining an average shape for a second set of training models comprising 3D models of the object type;
determining a decomposition of the second set of training models;
determining a relationship between the first set of training models and the second set of training models as a function of the average shape for the first set of training models, the decomposition of the first set of training models, the average shape for the second set of training models, and the decomposition of the second set of training models, each model in the first set of training models corresponding to a model in the second set of training models;
determining a relationship between a subject model corresponding to a 2D model corresponding to the object and the first set of training models; and
generating the digital 3D model of the object based on a predicted shape of the object based on the relationship between the first set of training models and the second set of training models and on the relationship between the subject model and the first set of training models.

### Claim 7 (depends on 6)

The computer implemented method of claim 6, wherein the first set of training models are derived from x-ray images, and wherein the subject model is derived from an x-ray image.

### Claim 8 (depends on 7)

The computer implemented method of claim 7, wherein the second set of training models are derived from one of CT scans and MRI scans.

### Claim 9 (depends on 6)

The computer implemented method of claim 6, wherein the second set of training models are derived from one of CT scans and MRI scans.

### Claim 10 (depends on 6)

The computer implemented method of claim 6, wherein determining the relationship between the subject model and the first set of training models comprises determining a fitting vector.

### Claim 11 (depends on 10)

The computer implemented method of claim 10, wherein generating the digital 3D model of the object based on the predicted shape is based at least in part on the fitting vector.

### Claim 12 (depends on 6)

The computer implemented method of claim 6, wherein generating the digital 3D model of the object based on the predicted shape comprises modifying the subject model.

### Claim 13

A non-transitory computer readable medium comprising computer executable instructions stored thereon which when executed by a processor cause a computer to perform a method of generating a digital 3D model of an object of an object type based on predicting a shape of the object, the method comprising:
determining an average shape for a first set of training models comprising 2D models of the object type;
determining a decomposition of the first set of training models;
determining an average shape for a second set of training models comprising 3D models of the object type;
determining a decomposition of the second set of training models;
determining a relationship between the first set of training models and the second set of training models as a function of the average shape for the first set of training models, the decomposition of the first set of training models, the average shape for the second set of training models, and the decomposition of the second set of training models, each model in the first set of training models corresponding to a model in the second set of training models;
determining a relationship between a subject model and the first set of training models, the subject model being a 2D model corresponding to the object; and
generating the digital 3D model of the object based on a predicted shape of the object based on the relationship between the first set of training models and the second set of training models and on the relationship between the subject model and the first set of training models.

### Claim 14 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the first set of training models are derived from x-ray images, and wherein the subject model is derived from an x-ray image.

### Claim 15 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the second set of training models are derived from one of CT scans and MRI scans.

### Claim 16 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the second set of training models are derived from one of CT scans and MRI scans.

### Claim 17 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein determining the relationship between the subject model and the first set of training models comprises determining a fitting vector.

### Claim 18 (depends on 17)

The non-transitory computer readable medium of claim 17, wherein generating the digital 3D model of the object based on the predicted shape is based at least in part on the fitting vector.

### Claim 19 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein generating the digital 3D model of the object based on the predicted shape comprises modifying the subject model.