Systems and methods for panoptic segmentation of images for autonomous driving

A system for panoptic image segmentation, the system comprising:

• a processor;
• and
• a non-transitory computer readable medium comprising programming instructions that when executed by the processor, will cause the processor to:
• receive an input image comprising a plurality of pixels;
• generate, from the input image, a semantic mask and an instance mask, wherein:
• the semantic mask comprises a single-channel mask that associates each pixel in the input image with a corresponding one of a plurality of labels, and
• the instance mask comprises a plurality of masks, wherein each of the plurality of masks:
• identifies an instance of a countable object in the input image, and
• is associated with an indication of whether that instance of the countable object is hidden behind another object in the input image;
• and
• combine the semantic mask and the instance mask to generate a panoptic mask for the input image,
• wherein the instance mask further comprises a plurality of bounding boxes, each of the plurality of bounding boxes being associated with an instance of one of a plurality of countable objects in the input image, the non-transitory computer readable medium further comprising programming instructions that when executed by the processor, will cause the processor to:
• determine the instance mask as invalid responsive to an overlap between a first bounding box and a second bounding box is less than a threshold indicative of a first countable object associated with the first bounding box is hidden behind a transparent amorphous region.

A computer program product comprising a memory and programming instructions that are configured to cause a processor to:

• receive an input image comprising a plurality of pixels;
• generate, from the input image, a semantic mask and an instance mask, wherein:
• the semantic mask comprises a single-channel mask that associates each pixel in the input image with a corresponding one of a plurality of labels, and
• the instance mask comprises a plurality of masks, wherein each of the plurality of masks:
• identifies an instance of a countable object in the input image, and
• is associated with an indication of whether that instance of the countable object is hidden behind another object in the input image;
• and
• combine the semantic mask and the instance mask to generate a panoptic mask for the input image.