Hierarchical constraint (hc)-based method and system for classifying fine-grained graptolite images

A hierarchical constraint (HC)-based method for classifying fine-grained graptolite images, comprising:

• step 1, collecting an original graptolite image;
• step 2, annotating fine granularity of a graptolite specimen in the original graptolite image;
• step 3, according to an annotation result of fine granularity, conducting cropping at a pixel level, cropping by an annotation box, and data enhancement on the original graptolite image to obtain a graptolite image representing the graptolite specimen, and constructing a graptolite dataset;
• step 4, extracting features in the graptolite image by a Convolutional Neural Network (CNN) model, which specifically comprises:
• extracting a feature map from an input graptolite image by convolution, activation and pooling operation of the CNN model to obtain a feature vector;
• and projecting, by an embedding layer, the feature vector into a feature with a dimension being a number of categories in the dataset, wherein a feature vector obtained after projection represents a prediction vector of the CNN to the input graptolite image, each value in the prediction vector represents a prediction score of a corresponding category of the graptolite image, and the higher the prediction score, the greater the probability that a graptolite in the graptolite image belongs to this category;
• step 5, calculating a similarity between graptolite images, and performing weighting according to a genetic relationship among species to obtain a weighted HC loss function (HC-Loss) of all graptolite images, which specifically comprises:
• dividing graptolite images in a same batch randomly into multiple groups of graptolite image pairs, each group of graptolite image pairs comprising two graptolite images;
• for two graptolite images in each graptolite image pair, quantifying a similarity weight according to the genetic relationship of categories graptolites in the two graptolite images belong, wherein for the graptolite images belonging to two categories, the closer the genetic relationship is, the greater the degree of similarity is, and the greater the similarity weight value is;
• on the contrary, the farther the genetic relationship is, the smaller the similarity weight value is;
• and
• in each training batch, calculating the weighted HC-Loss of all graptolite images by the following process:
• calculating a Euclidean distance between prediction vectors of two graptolite images in each graptolite image pair;
• according to the similarity weight value for each graptolite image pair, making weighted summation on the similarity of all groups of graptolite image pairs, and dividing a sum by the number of groups to obtain weighted HC-Loss of all graptolite images;
• step 6, calculating cross-entropy loss (CE-Loss) used to represent a difference between predicted probability distribution of the CNN model and real label distribution of images;
• step 7, taking a weighted sum of HC-Loss and CE-Loss as a total loss function of the CNN model in a training stage;
• and
• step 8, training the CNN model.