Differentiable maps for landmark localization

A computer-implemented method comprising:

• selecting an image from a set of images;
• determining, by processing the selected image using a detector portion of a machine learning model:
• a point cluster comprising points representing candidate keypoints in the selected image;
• and
• a set of uncertainties, each uncertainty corresponding to a respective point in the detected point cluster;
• obtaining a graph for the determined point cluster, the graph having edges, each edge linking a respective pair of points in the determined point cluster;
• generating, using the determined point cluster and the determined set of uncertainties, a differentiable map representing the point cluster and a spatial variation of uncertainty associated with the point cluster, wherein generating the differential map comprises:
• computing a plurality of edge map components, each edge map component corresponding to a respective edge of the graph and being a differentiable function of the uncertainties of the points linked by the respective edge of the graph;
• and
• combining the plurality of edge map components to generate the map;
• generating, by processing the differentiable map and a portion of the selected image using a decoder portion of the machine learning model, a candidate reconstructed image;
• and
• updating the machine learning model to reduce a loss function, the loss function comprising a term penalizing a difference between at least one representation of the selected image and a corresponding at least one representation of the candidate reconstructed image.

One or more non-transitory storage media comprising machine-readable instructions which, when executed by a computer, cause the computer to carry out operations comprising:

• selecting an image from a set of images;
• determining, by processing the selected image using a detector portion of a machine learning model:
• a point cluster comprising points representing candidate keypoints in the selected image;
• and
• a set of uncertainties, each uncertainty corresponding to a respective point in the detected point cluster;
• obtaining a graph for the determined point cluster, the graph having edges, each edge linking a respective pair of points in the determined point cluster;
• generating, using the determined point cluster and the determined set of uncertainties, a differentiable map representing the point cluster and a spatial variation of uncertainty associated with the point cluster, wherein generating the differential map comprises:
• computing a plurality of edge map components, each edge map component corresponding to a respective edge of the graph and being a differentiable function of the uncertainties of the points linked by the respective edge of the graph;
• and
• combining the plurality of edge map components to generate the map;
• generating, by processing the differentiable map and a portion of the selected image using a decoder portion of the machine learning model, a candidate reconstructed image;
• and
• updating the machine learning model to reduce a loss function, the loss function comprising a term penalizing a difference between at least one representation of the selected image and a corresponding at least one representation of the candidate reconstructed image.