Training a deep neural network model to generate rich object-centric embeddings of robotic vision data

A system comprising instructions stored in memory, and one or more processors executing the stored instructions to perform a method that includes:

• identifying a first image of an environment, the first image capturing objects in the environment and being captured from a first point of view;
• identifying a second image of the environment, the second image capturing at least a subset of the objects in the environment and being captured from a second point of view;
• processing the first image using an object recognition model to identify a plurality of first object regions in the first image, each of the first object regions corresponding to a respective one of the objects captured by the first image;
• generating a plurality of first object images based on the first object regions, each of the first object images being based on a respective one of the first object regions and capturing a respective one of the objects;
• processing the second image using the object recognition model to identify a plurality of second object regions in the second image, each of the second object regions corresponding to a respective one of the objects captured by the second image;
• generating a plurality of second object images based on the second object regions, each of the second object images being based on a respective one of the second object regions and capturing a respective one of the objects;
• processing each of the first object images using an object-contrastive model to generate respective first object image embeddings, wherein the object-contrastive model is a deep neural network model;
• processing each of the second object images using the object-contrastive model to generate respective second object image embeddings;
• generating a plurality of anchor, positive pairs that each include a respective one of the first object image embeddings and a respective one of second object image embeddings determined to be most similar to the respective one of the first object image embeddings;
• generating at least one metric loss based on contrasting the anchor, positive pairs with anchor, negative pairs that each include a respective one of the first object image embeddings and a respective one of the second object image embeddings not determined to be most similar to the respective one of the first object image embeddings;
• updating the object-contrastive model based on the at least one metric loss.