Generating masked regions of an image using a predicted user intent

A system comprising:

• a memory component;
• and
• a processing device coupled to the memory component, the processing device to perform operations comprising:
• receiving a user interaction corresponding to a region in an image;
• generating a feature vector using an encoder and the user interaction;
• generating a first classification using a first classifier and the feature vector;
• generating, a mask of the region using a decoder, the feature vector and the first classification;
• and
• outputting a masked region based on the generated mask.

A computer-readable medium storing instructions that, when executed by a processor, cause the processor to:

• receive a set of inputs including an image and at least one of a gesture map corresponding to a user interaction with a region in the image, or a distance map corresponding to a distance of the user interaction from the region in the image;
• generate a feature vector using an encoder based on the set of inputs;
• determine a mode type using the feature vector and a first classifier;
• determine a gesture type using the feature vector and a second classifier;
• generate, using a decoder, a mask of the region, based on the feature vector, and at least one of the mode type or the gesture type;
• and
• output a masked region based on the generated mask.