Open vocabulary instance segmentation with noise estimation and robust student

A method for training a machine learning model, comprising:

• receiving a training image and a caption for the training image, wherein the caption includes text describing an object in the training image;
• generating a pseudo mask for the object using a teacher network based on the text describing the object;
• generating a mask for the object using a student network;
• generating text features for the caption using a text encoder;
• and
• updating parameters of the student network based on the mask, the pseudo mask, and the text features.

An apparatus for training a machine learning model, comprising:

• A teacher network configured to generate a pseudo mask for an object in an image based on text describing the object in the image;
• a student network configured to generate a mask for the object;
• a noise estimation network configured to compute noise information for the image;
• and
• a training component configured to update parameters of the student network based on the mask, the pseudo mask, and the noise information.