US 8,478,052 B1Granted

Image classification

AssigneeGoogle Inc.

InventorsYangli Hector Yee, Samy Bengio, Charles Rosenberg, Erik Murphy-Chutorian

Life of this patent

Estimating the term…

Loading patent…

Abstract

An image classification system trains an image classification model to classify images relative to text appearing with the images. Training images are iteratively selected and classified by the image classification model according to feature vectors of the training images. An independent model is trained for unique n-grams of text. The image classification system obtains text appearing with an image and parses the text into candidate labels for the image. The image classification system determines whether an image classification model has been trained for the candidate labels. When an image classification model corresponding to a candidate label has been trained, the image classification subsystem classifies the image relative to the candidate label. The image is labeled based on candidate labels for which the image is classified as a positive image.

Claims26

Dependent claims

A computer-implemented method, comprising:

obtaining a plurality of n-grams, each of the n-grams including a unique set of one or more terms;
for each of the n-grams:
identifying, in a processing device, a plurality of training images for training an image classification model, the plurality of training images comprising:
positive training images having relevance measures, for the n-gram, that satisfy a relevance threshold;
and
negative training images having relevance measures, for the n-gram, that do not satisfy the relevance threshold;
selecting, in the processing device, a training image from the plurality of training images, wherein the selecting comprises semi-randomly selecting the training image subject to a selection requirement specifying that a second image be selected with a specified likelihood;
classifying, in the processing device, the training image with the image classification model based on a feature vector of the training image, the feature vector comprising image feature values for the training image;
and
training, in the processing device, the image classification model based on the feature vector of the training image and the classification of the training image.

7 more elements

A non-transitory computer readable medium encoded with a computer program comprising instructions that when executed operate to cause a computer to perform operations:

obtaining a plurality of n-grams, each of the n-grams including a unique set of one or more terms;
for each of the n-grams:
identifying a plurality of training images for training an image classification model, the plurality of training images comprising:
positive training images having relevance measures, for the n-gram, that satisfy a relevance threshold;
and
negative training images having relevance measures, for the n-gram, that do not satisfy the relevance threshold;
selecting a training image from the plurality of training images, wherein the selecting comprises semi-randomly selecting the training image subject to a selection requirement specifying that a second image be selected with a specified likelihood;
classifying the training image with the image classification model based on a feature vector of the training image, the feature vector comprising image feature values for the training image;
and
training the image classification model based on the feature vector of the training image and the classification of the training image.

7 more elements

Dependencies are inferred from the claim text.