Systems and methods for creating a visual vocabulary

A device for generating a visual vocabulary, the device comprising:

• one or more computer-readable media configured to store labeled images;
• and
• one or more processors that are coupled to the one or more computer-readable media and that are configured to cause the device to
• extract descriptors from one or more labeled images, wherein the labels include semantic information, and wherein extracted descriptors include visual information;
• augment the descriptors with semantic information from the labels;
• generate clusters of descriptors in an augmented space based on the semantic information and the visual information of the descriptors;
• generate a respective augmented-space classifiers for each one of the clusters of descriptors in the augmented space;
• generate clusters of descriptors in a descriptor space based on the clusters of descriptors in the augmented space, wherein two or more clusters of descriptors in the augmented space are associated with a corresponding cluster of descriptors in the descriptor space;
• and
• associate the two or more augmented-space classifiers for the two or more clusters of descriptors in the augmented space with the corresponding cluster of the clusters of descriptors in the descriptor space.

A method for encoding a descriptor, the method comprising:

• obtaining a descriptor;
• mapping the descriptor to a descriptor-space cluster in a descriptor space;
• applying a plurality of augmented-space classifiers that are associated with the descriptor-space cluster to the descriptor to generate respective augmented-space-classification scores;
• and
• generating a descriptor representation that includes the augmented-space-classification scores.