Process and system for three-dimensional modelling of tissue of a subject, and surgical planning process and system

A process for tissue modelling of a subject of one or more tissue type at a region of interest (ROI) of said subject, for forming a three-dimensional (3D) model of a region of interest (ROI) of a subject with one or more tissue types, said process including the steps of:

• (i) utilising a rule-based method to automatically generate a weak annotation from a midline slice of a subject in a 3 Dimensional (3D) medical image of said subject, wherein the rule-based method detects the approximate tissue locations from the midline slice, and, and further determines the initial seed areas;
• (ii) developing a neural network model to generate multi-scale feature maps and pixel classifications from the midline slice;
• (iii) utilising a clustering-based method to generate segmentation proposals based on multi-scale feature maps and the seed areas from (i) and (ii).
• (iv) further fine-tuning the proposal with several rule-based operations to explicitly embed within an anatomical prior of the region of interest (ROI), and wherein the seed areas are updated according to the fine-tuned proposal;
• and
• (v) training the neural network model with a comprehensive loss, which simultaneously optimizes the pixel classification and feature distribution of feature maps based on the proposals.

A system for tissue modelling of a subject of one or more tissue type at a region of interest (ROI) of said subject, for forming a three-dimensional (3D) model of a region of interest (ROI) of a subject with one or more tissue types, said system including an input module, a processor, a neutral network, and an output module, wherein:

• said input module receives a 3 Dimensional (3D) medical image containing a plurality of 2 Dimensional (2D) slices of a subject of a region of interest (ROI) of said subject;
• said processor and a neural network provide the process of:
• (i) using a rule-based method to automatically generate a weak annotation from the plurality of 2 Dimensional (2D) slices of said subject, wherein the rule-based method detects the approximate tissue locations from the plurality of 2 Dimensional (2D) slices, and further determines the initial seed areas;
• (ii) developing a neural network model to generate multi-scale feature maps and pixel classifications from the plurality of 2 Dimensional (2D) slices;
• (iii) utilising a clustering-based method to generate segmentation proposals based on multi-scale feature maps and the seed areas;
• (iv) further fine-tuning the proposal with several rule-based operations to explicitly embed within an anatomical prior of the region of interest (ROI), and wherein the seed areas are updated according to the fine-tuned proposal;
• and
• (i) training the neural network model with a comprehensive loss, which simultaneously optimizes the pixel classification and feature distribution of feature maps based on the proposals;
• and
• said output module provides an output representation a three-dimensional (3D) model of a region of interest (ROI) of the subject with one or more tissue types.