Open vocabulary image segmentation

A processor-implemented method, comprising:

• accessing an input image;
• processing the input image using an image encoder to generate an image embedding tensor;
• processing the image embedding tensor using a mask decoder machine learning model to generate a set of mask embedding tensors;
• processing a textual input using a text encoder to generate a text embedding tensor;
• and
• generating a set of augmented masks based on aggregating the text embedding tensor with the set of mask embedding tensors.

One or more non-transitory computer-readable media comprising processor-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to:

• access an input image;
• process the input image using an image encoder to generate an image embedding tensor;
• process the image embedding tensor using a mask decoder machine learning model to generate a set of mask embedding tensors;
• process a textual input using a text encoder to generate a text embedding tensor;
• and
• generate a set of augmented masks based on aggregating the text embedding tensor with the set of mask embedding tensors.

A processing system, comprising:

• means for accessing an input image;
• means for processing the input image using an image encoder to generate an image embedding tensor;
• means for processing the image embedding tensor using a mask decoder machine learning model to generate a set of mask embedding tensors;
• means for processing a textual input using a text encoder to generate a text embedding tensor;
• and
• means for generating a set of augmented masks based on aggregating the text embedding tensor with the set of mask embedding tensors.