Method and system for extracting sentiments or mood from art images

A system for extracting sentiments or mood from art images, the system comprising at least one processor,

• wherein the at least one processor is configured to:
• receive at least one of the art images as an input image, and preprocess the input image;
• extract features from the preprocessed input image, wherein the extracting the features comprises:
• predicting a color label corresponding to a dominant perceptual color detected from the preprocessed input image, wherein the predicting the color label comprises converting the preprocessed input image to a color space, applying clustering to the preprocessed input image on the color space to obtain at least three dominant colors classes representing three different color pixel values in the color space, respectively, determining a hue value and a cone angle based on the at least three dominant colors classes, detecting the dominant perceptual color based on a threshold range of the hue value and the cone angle, mapping the dominant perceptual color with a reference color label as defined in a database, and predicting the color label based on the mapping,
• detecting a dominant subject from the preprocessed input image, wherein the detecting the dominant subject comprises obtaining a plurality of classes related to the preprocessed input image, extracting art specific features for subject classification based on the plurality of classes, classifying the plurality of classes into a plurality of subject classes based on the art specific features, and predicting a dominant subject name based on the plurality of subject classes,
• detecting low-level image features from the preprocessed input image by extracting at least one of Local binary patterns, or Speeded-Up Robust Feature based on the preprocessed input image, and
• extracting mood feature information based on a description information included in the input image from a keyword present in the description information;
• input the extracted features to an artificial neural network to obtain a plurality of mood/sentiments classes corresponding to the extracted features;
• and
• predict at least one of a mood or a sentiment that is present in the input image based on the dominant perceptual color and the plurality of mood/sentiments classes,
• wherein the predicting the at least one of the mood or the sentiment comprises:
• mapping the dominant perceptual color and the low-level image features with respect to the plurality of mood/sentiments classes,
• obtaining a relationship between the dominant perceptual color, the low-level image features, and the plurality of mood/sentiments classes, respectively, based on the mapping, and
• predicting the at least one of the mood or the sentiment that is present in the input image based on the obtained relationship.

A non-transitory computer-readable storage medium storing at least one instruction which, when executed by at least one processor, causes the at least one processor to execute a method including:

• receiving at least one of art images as an input image;
• preprocessing the input image;
• extracting features from the preprocessed input image, wherein the extracting the features includes:
• predicting a color label corresponding to a dominant perceptual color detected from the preprocessed input image, wherein the predicting the color label comprises converting the preprocessed input image to a color space, applying clustering to the preprocessed input image on the color space to obtain at least three dominant colors classes representing three different color pixel values in the color space, respectively, determining a hue value and a cone angle based on the at least three dominant colors classes, detecting the dominant perceptual color based on a threshold range of the hue value and the cone angle, mapping the dominant perceptual color with a reference color label as defined in a database, and predicting the color label based on the mapping,
• detecting a dominant subject from the preprocessed input image, wherein the detecting the dominant subject comprises:
• obtaining a plurality of classes related to the preprocessed input image, extracting art specific features for subject classification based on the plurality of classes, classifying the plurality of classes into a plurality of subject classes based on the art specific features, and predicting a dominant subject name based on the plurality of subject classes,
• detecting low-level image features from the preprocessed input image by extracting at least one of Local binary patterns, or Speeded-Up Robust Feature based on the preprocessed input image, and
• extracting mood feature information based on a description information included in the input image from a keyword present in the description information;
• inputting the extracted features to an artificial neural network to obtain a plurality of mood/sentiments classes corresponding to the extracted features;
• and
• predicting at least one of a mood or a sentiment that is present in the input image based on the dominant perceptual color and the plurality of mood/sentiments classes,
• wherein the predicting the at least one of the mood or the sentiment comprises:
• mapping the dominant perceptual color and the low-level image features with respect to the plurality of mood/sentiments classes,
• obtaining a relationship between the dominant perceptual color, the low-level image features, and the plurality of mood/sentiments classes, respectively, based on the mapping, and
• predicting the at least one of the mood or the sentiment that is present in the input image based on the obtained relationship.