Automatic detection of lesions in medical images using 2D and 3D deep learning networks

An apparatus comprising:

• means for receiving a 3D input medical image depicting one or more lesions;
• means for segmenting the one or more lesions from one or more 2D slices extracted from the 3D input medical image using a trained 2D segmentation network;
• means for extracting first probability values from results of the segmentation of the one or more lesions from the one or more 2D slices;
• means for segmenting the one or more lesions from a 3D patch extracted from the 3D input medical image using a trained 3D segmentation network;
• means for extracting second probability values from results of the segmentation of the one or more lesions from the 3D patch;
• means for fusing the first probability values and the second probability values using a trained machine learning based fusion model to generate final segmentation results, wherein the trained machine learning based fusion model is trained based on 1) probability values extracted from results of a segmentation of 2D training slices, 2) probability values extracted from results of a segmentation of 3D training images, and 3) ground truth annotations of the probability values extracted from the results of the segmentation of 2D training slices and the 3D training images;
• and
• means for outputting the final segmentation results.

A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:

• receiving a 3D input medical image depicting one or more lesions;
• segmenting the one or more lesions from one or more 2D slices extracted from the 3D input medical image using a trained 2D segmentation network;
• extracting first probability values from results of the segmentation of the one or more lesions from the one or more 2D slices;
• segmenting the one or more lesions from a 3D patch extracted from the 3D input medical image using a trained 3D segmentation network;
• extracting second probability values from results of the segmentation of the one or more lesions from the 3D patch;
• fusing the first probability values and the second probability values using a trained machine learning based fusion model to generate final segmentation results, wherein the trained machine learning based fusion model is trained based on 1) probability values extracted from results of a segmentation of 2D training slices, 2) probability values extracted from results of a segmentation of 3D training images, and 3) ground truth annotations of the probability values extracted from the results of the segmentation of 2D training slices and the 3D training images;
• and
• outputting the final segmentation results.