Unique sampling of objects in image datasets

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• determining a plurality of embeddings for a plurality of objects depicted in a plurality of images in the dataset, wherein a size of the plurality of embeddings exceeds a size of one or more memory resources associated with one or more instances of a processing engine;
• generating a tree structure within the one or more memory resources with the plurality of embeddings by:
• dividing the plurality of embeddings into a plurality of subsets of embeddings that are each smaller in size than the size of the one or more memory resources;
• for each of the plurality of subsets of embeddings,
• generating a first node that stores a first subset of embeddings included in the plurality of subsets of embeddings;
• and
• generating a first plurality of nodes as children of the first node, wherein each node included in the first plurality of nodes stores embeddings included in the first subset of embeddings and corresponds to a different region of a latent space in which the plurality of embeddings are positioned, and combining a plurality of nodes generated for the plurality of subsets of embeddings to construct the tree structure;
• sampling a subset of sampled embeddings from the plurality of embeddings via a traversal of the tree structure, wherein sampling the subset of sampled embeddings comprises sampling a first embedding from the first node, and, in response to sampling the first embedding:
• marking the first node and one or more descendant nodes of the first node that include the first embedding as sampled, and adding a first image that depicts a first object represented by the first embedding to the subset of images;
• and
• generating a sampled dataset that includes a subset of images from the plurality of images based on the subset of sampled embeddings and a number of images to be included in the sampled dataset.

A system, comprising:

• one or more memories that store instructions, and
• one or more processors that are coupled to the one or more memories and,
• when executing the instructions, are configured to perform the steps of:
• determining a plurality of embeddings for a plurality of objects depicted in a plurality of images in the dataset, wherein a size of the plurality of embeddings exceeds a size of one or more memory resources associated with one or more instances of a processing engine;
• generating a tree structure within the one or more memory resources with the plurality of embeddings by:
• dividing the plurality of embeddings into a plurality of subsets of embeddings that are each smaller in size than the size of the one or more memory resources;
• for each of the plurality of subsets of embeddings,
• generating a first node that stores a first subset of embeddings included in the plurality of subsets of embeddings;
• and
• generating a first plurality of nodes as children of the first node, wherein each node included in the first plurality of nodes stores embeddings included in the first subset of embeddings and corresponds to a different region of a latent space in which the plurality of embeddings are positioned, and
• combining a plurality of nodes generated for the plurality of subsets of embeddings to construct the tree structure;
• sampling a subset of sampled embeddings from the plurality of embeddings via a traversal of the tree structure, wherein sampling the subset of sampled embeddings comprises sampling a first embedding from the first node, and in response to sampling the first embedding:
• marking the first node and one or more descendant nodes of the first node that include the first embedding as sampled, and adding a first image that depicts a first object represented by the first embedding to the subset of images;
• and
• generating a sampled dataset that includes a subset of images from the plurality of images based on the subset of sampled embeddings and a number of images to be included in the sampled dataset.