System and method for multi-scale coarse-to-fine segmentation of images to detect pancreatic ductal adenocarcinoma

A method comprising:

• obtaining a first image that depicts a first volume of voxels;
• performing coarse segmentation of the first image using a first neural network trained (i) to process images having the first volume of voxels and (ii) to produce first output data representing a first likelihood that the first image includes more than a threshold number of voxels that depict a tumor;
• determining a region of interest of the first image based on the coarse segmentation;
• performing multi-stage fine segmentation using a plurality of images that are each generated based on the region of interest of the first image, comprising:
• performing a first fine stage segmentation of a second image (i) including a second volume of voxels that is less than the first volume of voxels and (ii) generated based on the region of interest of the first image using a second neural network trained (i) to process images having the second volume of voxels and (ii) to produce second output data representing a second likelihood that the second image includes more than a threshold number of voxels that depict a tumor;
• and
• performing a second fine stage segmentation of a third image (i) including a third volume of voxels that is less than the second volume of voxels and (ii) generated based on the region of interest of the first image using a third neural network trained (i) to process images having the third volume of voxels and (ii) to produce third output data representing a third likelihood that the third image includes more than a threshold number of voxels that depict a tumor;
• and
• determining, based on the first output data, the second output data, and the third output data whether the obtained first image depicts a tumor.

A computer-readable storage device having stored thereon instructions, which, when executed by data processing apparatus, cause the data processing apparatus to perform operations, the operations comprising:

• obtaining a first image that depicts a first volume of voxels;
• performing coarse segmentation of the first image using a first neural network trained (i) to process images having the first volume of voxels and (ii) to produce first output data representing a first likelihood that the first image includes more than a threshold number of voxels that depict a tumor;
• determining a region of interest of the first image based on the coarse segmentation;
• performing multi-stage fine segmentation using a plurality of images that are each generated based on the region of interest of the first image, comprising:
• performing a first fine stage segmentation of a second image (i) including a second volume of voxels that is less than the first volume of voxels and (ii) generated based on the region of interest of the first image using a second neural network trained (i) to process images having the second volume of voxels and (ii) to produce second output data representing a second likelihood that the second image includes more than a threshold number of voxels that depict a tumor;
• and
• performing a second fine stage segmentation of a third image (i) including a third volume of voxels that is less than the second volume of voxels and (ii) generated based on the region of interest of the first image using a third neural network trained (i) to process images having the third volume of voxels and (ii) to produce third output data representing a third likelihood that the third image includes more than a threshold number of voxels that depict a tumor;
• and
• determining, based on the first output data, the second output data, and the third output data whether the obtained first image depicts a tumor.