System and method for measuring image similarity based on semantic meaning

A computer implemented method for characterizing images and determining image similarity based on semantic meaning of images, comprising:

• for each semantic category of a plurality of semantic categories, forming a set of perceptual features comprising required features and frequently occurring features, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories;
• determining features of an image;
• comparing features of the image to the sets of perceptual features of the semantic categories;
• and
• classifying the image as belonging to one of the semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in the features of the image.

An apparatus comprising:

• means for forming, for each semantic category of a plurality of semantic categories, a set of perceptual features comprising required features and frequently occurring features, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories;
• means for determining features of an image;
• means for comparing features of the image to the sets of perceptual features of the semantic categories;
• and
• means for classifying the image as belonging to one of the semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in the features of the image.

A computer implemented method for characterizing images and determining image similarity based on semantic meaning of images, comprising:

• performing subjective experiments with human observers to determine perceptual features of a plurality of images and to determine sets of semantic categories for the plurality of images, wherein each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories;
• for each semantic category of a plurality of semantic categories, forming a set of perceptual features comprising required features and frequently occurring features;
• determining features of a query image;
• classifying the query image as belonging to one of the semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in the features of the image, wherein classifying further comprises computing a similarity measure between the perceptual features used to describe a semantic category and corresponding features determined from the query image;
• and
• assigning the query image to that semantic category that corresponds to a highest value of the similarity measure.

A computer implemented method for characterizing images and determining image similarity based on semantic meaning of images, comprising:

• performing subjective experiments with human observers to determine perceptual features of a plurality of images and to determine sets of semantic categories for the plurality of images, wherein each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories;
• for each semantic category of a plurality of semantic categories, forming a set of perceptual features comprising required features and frequently occurring features;
• determining features of a query image;
• classifying the query image as belonging to one of the semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in the features of the image, wherein classifying further comprises computing a similarity measure between the perceptual features used to describe a semantic category and corresponding features determined from the query image, where the similarity measure comprises:
• sim
• ⁡
• (
• x
• ,
• c
• i
• )
• =
• 1
• N
• i
• ⁢
• ∏
• j
• =
• 1
• M
• i
• ⁢
• ⁢
• τ
• (
• RF
• j
• ⁡
• (
• x
• ⁢
• 
• c
• i
• )
• ,
• RF
• j
• ⁡
• (
• c
• i
• )
• )
• ·
• ∑
• j
• =
• 1
• N
• i
• ⁢
• ⁢
• τ
• (
• FO
• j
• ⁡
• (
• x
• ⁢
• 
• c
• i
• )
• ,
• FO
• j
• ⁡
• (
• c
• i
• )
• )
• ,
• where the query image is x, a semantic category is ci, {RFj(ci)|j=1, . . . , Mi} is a set of Mi required features, {FOj(ci)|j=1, . . . , Ni} is a set of Ni frequently occurring features,
• τ
• ⁡
• (
• a
• ,
• B
• )
• =
• {
• 1
• ,
• (
• ∃
• i
• )
• 0
• ,
• (
• ∀
• i
• )
• ⁢
• a
• =
• b
• i
• a
• ≠
• b
• i
• ,
• and
• ⁢
• ⁢
• B
• =
• {
• b
• i
• }
• i
• =
• 1
• ,
• ⁢
• …
• ⁢
• ,
• l
• ;
• and
• assigning the query image to that semantic category that corresponds to a highest value of the similarity measure.