System and methods for automatic image alignment of three-dimensional image

An image processing system, comprising:

• a display device;
• a non-transitory memory including instructions;
• and
• a processor, wherein, when executing the instructions, the processor causes the image processing system to:
• receive a three-dimensional (3D) image volume, an imaging modality of the 3D image volume including at least one of computed tomography (CT), magnetic resonance imaging (MRI), and ultrasound;
• pre-process the 3D image volume by selecting a sub-volume of interest;
• detect a plurality of landmarks in the sub-volume of interest using a deep neural network trained to identify anatomical landmarks;
• estimate transformation parameters based on the plurality of landmarks to adjust rotation angles and translation of the sub-volume of interest;
• adjust rotation angles and translation of the sub-volume of interest using the transformation parameters to produce a first aligned sub-volume of interest;
• determine confidence in the transformation parameters based on the first aligned sub-volume of interest;
• refine the transformation parameters iteratively if the confidence is below a predetermined threshold;
• adjusting rotation angles and translation of the first aligned sub-volume of interest using the refined transformation parameters to produce a second aligned sub-volume of interest;
• determining confidence in the refined transformation parameters based on the second aligned sub-volume of interest by:
• projecting the second aligned sub-volume of interest onto a plurality of projection planes to produce a plurality of projection images;
• and
• mapping the plurality of projection images to a corresponding plurality of confidence scores using a confidence estimation network;
• and
• determining confidence in the refined transformation parameters through majority voting using the plurality of confidence scores corresponding to the plurality of projection images.

A method comprising:

• receiving a three-dimensional (3D) image volume comprising a first plurality of two-dimensional (2D) images parallel to a first plane, an imaging modality of the 3D image volume including at least one of computed tomography (CT), magnetic resonance imaging (MRI), and ultrasound;
• pre-processing the 3D image volume by selecting a subset of the first plurality of 2D images to produce a sub-volume of interest including an anatomical region of interest;
• detecting at least three landmarks in the sub-volume of interest using a deep neural network trained to identify anatomical landmarks;
• estimating transformation parameters, including an anatomical plane of interest intersecting the sub-volume of interest, based on the at least three landmarks;
• performing multiplanar reformatting of the sub-volume of interest along the anatomical plane of interest to produce a first aligned sub-volume of interest, wherein the first aligned sub-volume of interest comprises a second plurality of 2D images parallel to the anatomical plane of interest;
• determining confidence in the transformation parameters based on the first aligned sub-volume of interest;
• refining the transformation parameters iteratively if the confidence is below a predetermined threshold;
• adjusting rotation angles and translation of the first aligned sub-volume of interest using the refined transformation parameters to produce a second aligned sub-volume of interest;
• determining confidence in the refined transformation parameters based on the second aligned sub-volume of interest by:
• projecting the second aligned sub-volume of interest onto a plurality of projection planes to produce a plurality of projection images;
• and
• mapping the plurality of projection images to a corresponding plurality of confidence scores using a confidence estimation network;
• and
• determining confidence in the refined transformation parameters through majority voting using the plurality of confidence scores corresponding to the plurality of projection images.