Method and system for machine learning classification based on structure or material segmentation in an image

A system for classifying a structure or material in an image of a subject, comprising:

• a segmenter configured to segment an image into one or more segmentations that correspond to respective structures or materials in the image, and to generate from the segmentations one or more segmentation maps of the image including categorizations of pixels or voxels of the segmentation maps assigned from one or more respective predefined sets of categories;
• a classifier that implements a trained classification machine learning model configured to generate, based on the segmentations maps, one or more classifications and to assign to the classifications respective scores indicative of a likelihood that the structure or material, or the subject, falls into the respective classifications;
• and
• an output for outputting a result indicative of the classifications and scores;
• wherein the segmenter comprises:
• a structure segmenter configured to generate structure segmentation maps including categorizations of the pixels or voxels assigned from a predefined set of structure categories and to employ a structure segmentation machine learning model to generate the structure segmentation maps,
• a material segmenter configured to generate material segmentation maps including categorizations of the pixels or voxels assigned from a predefined set of material categories and to employ a material segmentation machine learning model to generate the material segmentation maps, and/or
• an abnormality segmenter configured to generate abnormality segmentation maps including categorizations of the pixels or voxels assigned from a predefined set of abnormality or normality categories and to employ an abnormality segmentation model to generate the abnormality segmentation maps.