System and method for semantic segmentation using Gaussian random field network

A system for semantic segmentation of an image, comprising:

• at least one non-transitory computer readable memory to store the image and a semantically segmented image;
• and
• a processor to perform a semantic segmentation of the image using a Gaussian random field (GRF) network to produce the semantically segmented image, wherein the GRF network is a neural network including
• a first subnetwork for determining a unary energy of each pixel in an image;
• a second subnetwork for determining pairwise energy of at least some pairs of pixels of the image;
• and
• a third subnetwork for determining an inference on a Gaussian random field (GRF) minimizing an energy function including a combination of the unary energy and the pairwise energy to produce a GRF inference defining probabilities of semantic labels for each pixel in the image;
• wherein the processor converts the image into a semantically segmented image by assigning to a pixel in the semantically segmented image a semantic label having the highest probability for a corresponding pixel in the image among the probabilities determined by the third subnetwork.

A non-transitory computer-readable medium with instructions stored thereon, that when executed by a processor, perform the steps comprising:

• determining unary energy of each pixel in an image using a first subnetwork;
• determining pairwise energy of at least some pairs of pixels of the image using a second subnetwork;
• determining, using a third subnetwork, an inference on a Gaussian random field (GRF) minimizing an energy function including a combination of the unary energy and the pairwise energy to produce a GRF inference defining probabilities of semantic labels for each pixel in the image;
• and
• converting the image into a semantically segmented image by assigning to a pixel in the semantically segmented image a semantic label having the highest probability for a corresponding pixel in the image among the probabilities determined by the third subnetwork, wherein the first subnetwork, the second subnetwork, and the third subnetwork are jointly trained as parts of a neural network.