Multiple cluster instance learning for image classification
AssigneeMicrosoft Technology Licensing, LLC
InventorsZhuowen Tu, Yan Xu, Junyan Zhu, Eric I-Chao Chang
Life of this patent
Estimating the term…
Abstract
The techniques and systems described herein create and train a multiple clustered instance learning (MCIL) model based on image features and patterns extracted from training images. The techniques and systems separate each of the training images into a plurality of instances (or patches), and then learn multiple instance-level classifiers based on the extracted image features. The instance-level classifiers are then integrated into the MCIL model so that the MCIL model, when applied to unclassified images, can perform image-level classification, patch-level clustering, and pixel-level segmentation.
Claims20
A method comprising:
- receiving a plurality of training images, each training image including one or more objects associated with a high-level image classification;
- separating each of the plurality of training images into a plurality of instances;
- extracting image features from each of the plurality of instances in each of the plurality of training images;
- training, via one or more processors configured with executable instructions, multiple instance-level classifiers based on the extracted image features, wherein each of the multiple instance-level classifiers are trained to associate an instance with one of the one or more objects, and wherein each of the multiple instance-level classifiers includes a learned weight based on a probability that the instance is associated with the one of the one or more objects;
- implementing the multiple instance-level classifiers in an image classification model;
- receiving at least one query search term;
- receiving a collection of unclassified images from an image-based search using the at least one query search term;
- and
- applying the image classification model to the collection of unclassified images.
One or more computer storage media storing computer-executable instructions that, when executed by one or more processors, perform operations comprising:
- receiving input that provides a high-level image classification;
- receiving at least one query search term;
- receiving a plurality of unclassified images from an image-based search using the at least one query search term;
- applying an image classification model to the plurality of unclassified images, the image classification model including multiple cluster classifiers that associate image patches with a plurality of different objects associated with the input, wherein individual ones of the multiple cluster classifiers include a learned weight based on a probability that individual ones of the image patches are associated with one of the plurality of different objects;
- and
- identifying a portion of the plurality of unclassified images that include at least one of the plurality of different objects associated with the input.
A system comprising:
- one or more processors;
- and
- one or more memories, coupled to the one or more processors, storing:
- an image access module, operable on the one or more processors, that receives a plurality of training images, each training image including one or more objects associated with a high-level image classification;
- a classifier learning module, operable on the one or more processors, that learns multiple instance-level classifiers based on features extracted from each of the plurality of training images, each instance-level classifier configured to associate an image patch with one of the one or more objects based at least in part on a learned weight based on a probability that the image patch is associated with the one of the one or more objects;
- an input/output module, operable on the one or more processors, that receives at least one query search term and receives a collection of unclassified images from an image-based search using the at least one query search term;
- and
- a model application module, operable on the one or more processors, that applies the multiple instance-level classifiers to the collection of unclassified images to discover a subset of images that contain the one or more objects.
Dependencies are inferred from the claim text.