Unsupervised representation learning with contrastive prototypes
Assigneesalesforce.com, inc.
InventorsJunnan Li, Chu Hong Hoi
Life of this patent
Estimating the term…
Abstract
The system and method are directed to a prototypical contrastive learning (PCL). The PCL explicitly encodes the hierarchical semantic structure of the dataset into the learned embedding space and prevents the network from exploiting low-level cues for solving the unsupervised learning task. The PCL includes prototypes as the latent variables to help find the maximum-likelihood estimation of the network parameters in an expectation-maximization framework. The PCL iteratively performs an E-step for finding prototypes with clustering and M-step for optimizing the network on a contrastive loss.
Claims20
A method for training a prototypical contrastive learning (PCL) framework to cluster images, comprising:
- receiving images at a momentum encoder of the PCL framework;
- determining, using the momentum encoder, features from the images;
- clustering, using the momentum encoder, the images into clusters according to the features;
- determining at least one prototype and at least one concentration of the clusters;
- determining, a contrastive loss function from the at least one prototype and the at least one concentration of the clusters;
- training an encoder of the PCL framework using the contrastive loss function and a subset of images in the images, wherein the training updates the weights of the encoder based on the contrastive loss function;
- updating weights of the momentum encoder using weights of the encoder;
- and
- determining second clusters from second images using the updated momentum encoder.
A system for training a prototypical contrastive learning (PCL) framework to cluster images, comprising:
- at least one memory configured to store the PCL framework and images;
- and
- a processor coupled to the memory and configured to execute instructions for training the PCL framework, the instructions comprising:
- receiving the images at a momentum encoder of the PCL framework;
- determining, using the momentum encoder, features from the images;
- clustering, using the momentum encoder, the images into clusters according to the features;
- determining at least one prototype and at least one concentration of the clusters;
- determining, a contrastive loss function from the at least one prototype and the at least one concentration of the clusters;
- training an encoder of the PCL framework using the contrastive loss function and a subset of images in the images;
- updating weights of the momentum encoder using weights of the encoder;
- and
- determining second clusters from second images using the updated momentum encoder.
A non-transitory computer-readable medium storing instructions thereon, that when executed by a processor, cause the processor to perform operations that train a prototypical contrastive learning (PCL) framework to cluster images, the operations comprising:
- receiving images at a momentum encoder of the PCL framework;
- determining, using the momentum encoder, features from the images;
- clustering, using the momentum encoder, the images into clusters according to the features;
- determining at least one prototype and at least one concentration of the clusters;
- determining, a contrastive loss function from the at least one prototype and the at least one concentration of the clusters;
- training an encoder of the PCL framework using the contrastive loss function and a subset of images in the images, wherein the training updates the weights of the encoder based on the contrastive loss function;
- updating weights of the momentum encoder using weights of the encoder;
- and
- determining second clusters from second images using the updated momentum encoder.
Dependencies are inferred from the claim text.