Image retrieval with deep local feature descriptors and attention-based keypoint descriptors

A computer-implemented method to train a machine-learned image descriptor model, the method comprising:

• performing, by a computing system comprising one or more computing devices, a first training process, the first training process comprising, for each of one or more first training images:
• processing, by the computing system, the first training image using a feature extraction portion of the machine-learned image descriptor model to obtain, as an output of the feature extraction portion, a first plurality of local feature descriptors for the first training image;
• evaluating, by the computing system, a first loss function based at least in part on the first plurality of local feature descriptors for the first training image;
• and
• training, by the computing system, the feature extraction portion of the machine-learned image descriptor model based at least in part on the first loss function;
• and
• performing, by the computing system, a second training process, the second training process comprising, for each of one or more second training images:
• processing, by the computing system, the second training image using the feature extraction portion of the machine-learned image descriptor model to obtain, as an output of the feature extraction portion, a second plurality of local feature descriptors for the first training image;
• processing the second plurality of local feature descriptors for the second training image using an attention portion of the machine-learned image descriptor model to obtain, as an output of the attention portion, a plurality of attention scores respectively for the second plurality of local feature descriptors;
• evaluating, by the computing system, a second loss function based at least in part on the plurality of attention scores for the second training image;
• and
• training, by the computing system, the attention portion of the machine-learned image descriptor model based at least in part on the second loss function.

One or more non-transitory computer-readable media that collectively store:

• a machine-learned image descriptor model configured to determine a set of keypoint descriptors for an input image, the machine-learned image descriptor model comprising:
• a feature extraction portion configured to extract a plurality of local feature descriptors from the input image;
• an attention portion configured to determine a plurality of attention scores respectively for the plurality of local feature descriptors;
• and
• a keypoint selection portion configured to select a plurality of keypoint descriptors from the plurality of local feature descriptors based at least in part on the plurality of attention scores, the plurality of keypoint descriptors corresponding to a subset of the plurality of local feature descriptors.