US 9,830,526 B1Granted

Generating image features based on robust feature-learning

AssigneeAdobe Systems Incorporated

InventorsZhe Lin, Xiaohui Shen, Jonathan Brandt, Jianming Zhang

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques for increasing robustness of a convolutional neural network based on training that uses multiple datasets and multiple tasks are described. For example, a computer system trains the convolutional neural network across multiple datasets and multiple tasks. The convolutional neural network is configured for learning features from images and accordingly generating feature vectors. By using multiple datasets and multiple tasks, the robustness of the convolutional neural network is increased. A feature vector of an image is used to apply an image-related operation to the image. For example, the image is classified, indexed, or objects in the image are tagged based on the feature vector. Because the robustness is increased, the accuracy of the generating feature vectors is also increased. Hence, the overall quality of an image service is enhanced, where the image service relies on the image-related operation.

Claims20

Dependent claims

A computer-implemented method for increasing robustness of a convolutional neural network that supports at least one of image classification, image tagging, or image retrieval, the computer-implemented method comprising:

accessing, by a computer system, a first training dataset comprising first image data, the first training dataset associated with a first image-related task and a first label applicable to the first image data;
accessing, by the computer system, a second training dataset comprising second training data, the second training dataset associated with a second task and a second label;
training, by the computer system, the convolutional neural network by at least:
minimizing a first loss function for the first training dataset based on the first image-related task and a second loss function for the second training dataset based on the second task, and
updating parameters of the convolutional neural network based on the minimizing of the first loss function and the second loss function;
inputting, by the computer system, image data of an image to the convolutional neural network;
and
generating, by the computer system, a feature vector for the image based on the parameters of the convolutional neural network, the feature vector enabling an application of an image-related operation to the image, the image-related operation comprising at least one of the image classification, the image tagging, or the image retrieval.

5 more elements

A computer-readable storage medium storing instructions that, when executed on a computing device, configure the computing device to perform operations comprising:

accessing a first training dataset comprising first image data, the first training dataset associated with a first image-related task and a first label applicable to the first image data;
accessing a second training dataset comprising second image data, the second training dataset associated with a second image-related task and a second label applicable to the second image data;
training a convolutional neural network by at least:
minimizing a first loss function for the first training dataset based on the first-image related task and a second loss function for the second training dataset based on the second-image related task, and
updating parameters of the convolutional neural network based on the minimizing of the first loss function and the second loss function;
inputting image data of an image to the convolutional neural network;
and
generating a feature vector for the image based on the parameters of the convolutional neural network, the feature vector enabling an application of an image-related operation to the image.

5 more elements

Dependencies are inferred from the claim text.