Region metrics for class balancing in machine learning systems

A method for image understanding comprising:

• receiving, by an input device, training data comprising an image comprising a plurality of pixels, each pixel of the plurality of pixels labeled with a class of a plurality of classes;
• applying, by a machine learning system of a computation engine executed by processing circuitry, a machine learning model to perform image understanding of each pixel of the plurality of pixels to determine an estimated class of the plurality of classes to which the pixel belongs;
• determining, by the machine learning system and based on the classes with which the plurality of pixels are labeled and the estimated classes of the plurality of pixels, a corresponding cross entropy loss for each class of the plurality of classes;
• for each class of the plurality of classes, determining, by the machine learning system and based on one or more region metrics indicative of a performance of the machine learning model in identifying pixels as belonging to the class, a corresponding weight for the class;
• adjusting, by the machine learning system and for each class of the plurality of classes, the cross entropy loss for the class with the corresponding weight determined for the class to obtain a corresponding weighted cross entropy loss for the class;
• and
• updating, by the machine learning system, the machine learning model with the corresponding weighted cross entropy loss for each class of the plurality of classes to improve a performance metric of the machine learning model for each class of the plurality of classes.

A non-transitory, computer-readable medium comprising instructions for causing processing circuitry of an image understanding system to:

• receive training data comprising an image comprising a plurality of pixels, each pixel of the plurality of pixels labeled with a class of a plurality of classes;
• and
• execute a machine learning system configured to:
• apply a machine learning model to perform image understanding of each pixel of the plurality of pixels to determine an estimated class of the plurality of classes to which the pixel belongs;
• determine, based on the classes with which the plurality of pixels are labeled and the estimated classes of the plurality of pixels, a corresponding cross entropy loss for each class of the plurality of classes;
• for each class of the plurality of classes, determine, based on one or more region metrics indicative of a performance of the machine learning model in identifying pixels as belonging to the class, a corresponding weight for the class;
• adjust, for each class of the plurality of classes, the cross entropy loss for the class with the corresponding weight determined for the class to obtain a corresponding weighted cross entropy loss for the class;
• and
• update the machine learning model with the corresponding weighted cross entropy loss for each class of the plurality of classes to improve a performance metric of the machine learning model for each class of the plurality of classes.