US 11,887,354 B2 · Weakly supervised image semantic segmentation method, system and apparatus based on intra-class discriminator

Abstract

A weakly supervised image semantic segmentation method based on an intra-class discriminator includes: constructing two levels of intra-class discriminators for each image-level class to determine whether pixels belonging to the image class belong to a target foreground or a background, and using weakly supervised data for training; generating a pixel-level image class label based on the two levels of intra-class discriminators, and generating and outputting a semantic segmentation result; and further training an image semantic segmentation module or network by using the label to obtain a final semantic segmentation model for an unlabeled input image. By means of the new method, intra-class image information implied in a feature code is fully mined, foreground and background pixels are accurately distinguished, and performance of a weakly supervised semantic segmentation model is significantly improved under the condition of only relying on an image-level annotation.

A weakly supervised image semantic segmentation method based on an intra-class discriminator, comprising:

extracting a feature image of a to-be-processed image through a feature extraction network, and obtaining an image semantic segmentation result of the to-be-processed image through an image semantic segmentation module, wherein the image semantic segmentation module is obtained through training based on a training image set and corresponding accurate pixel-level class labels;
wherein, the corresponding accurate pixel-level class labels are obtained through a first intra-class discriminator and a second intra-class discriminator based on the training image set and corresponding image-level class labels;
the first intra-class discriminator and the second intra-class discriminator are separately constructed based on a deep network, and a method for training the first intra-class discriminator and the second intra-class discriminator comprises:
step S10:
extracting a feature image of each image in the training image set through the feature extraction network to obtain a training feature image set, and constructing a first loss function of the first intra-class discriminator and a second loss function of the second intra-class discriminator, respectively;
step S20:
training the first intra-class discriminator based on the training feature image set, the corresponding image-level class labels and the first loss function to obtain preliminary pixel-level foreground and background labels corresponding to all classes of each image in the training image set, wherein step S20 further comprises:
step S21:
for each image-level class label c of each feature image in the training feature image set, setting a direction vector wc, using a pixel in a direction of the direction vector wc as a foreground pixel of a class c, and using a pixel in an opposite direction of the direction vector wcas a background pixel of the class c;
step S22:
calculating a first loss value based on the direction vector w, and the training feature image set, and updating wc based on the first loss value;
and
step S23:
repeatedly performing step S21 and step S22 until a set first quantity of times of training is reached, wherein a trained first intra-class discriminator and the preliminary pixel-level foreground and background labels corresponding to all the classes of each image in the training image set are obtained;
step S30:
training the second intra-class discriminator based on the training feature image set, the corresponding preliminary pixel-level foreground and background labels and the second loss function to obtain accurate pixel-level foreground and background labels corresponding to all the classes of each image in the training image set;
and
step S40:
generating the accurate pixel-level class labels based on the accurate pixel-level foreground and background labels corresponding to all the classes of each image in the training image set and the corresponding image-level class labels.

16 more elements

Weakly supervised image semantic segmentation method, system and apparatus based on intra-class discriminator

Life of this patent

Weakly supervised image semantic segmentation method, system and apparatus based on intra-class discriminator

Life of this patent

Abstract

Claims17