Universal image representation based on a multimodal graph

A method performed by one or more computing systems for training an autoencoder to generate a universal image representation of a target image, the method comprising:

• for each of a plurality of autoencoder training images,
• identifying segments of the autoencoder training image, each segment associated with an attribute;
• generating a graph that includes vertices and edges, each vertex representing a segment and each edge representing a relationship between the segments represented by the vertices connected by the edge;
• and
• generating subgraphs for vertices that include a home vertex and neighboring vertices of the home vertex representing segments of the target image;
• and
• training the autoencoder using the subgraphs as training data, the autoencoder for generating latent variables, wherein the universal image representation of the target image is derived from the latent variables.

A method performed by one or more computing systems for generating a representation of a target image having regions of interest labeled with attributes, the method comprising:

• identifying segments of the target image that correspond to regions of interest;
• generating a graph that includes vertices and edges, each vertex representing a segment and each edge representing a relationship between the segments represented by the vertices connected by the edge;
• generating a subgraph for each vertex that includes the vertex as a home vertex and neighboring vertices representing segments of the target image;
• and
• apply an autoencoder to each subgraph to generate latent variables to represent the subgraph, wherein the representation of the target image is based on the latent variables of the subgraph.

One or more computing systems for training an autoencoder to generate a representation of a target image, the one or more computing systems comprising:

• one or more computer-readable storage mediums for storing computer-executable instructions for controlling the one or more computing systems to:
• for each of a plurality of autoencoder training images:
• identify segments of the autoencoder training image, each segment associated with an attribute;
• generate a graph that includes vertices and edges, each vertex representing a segment and each edge representing a relationship between the segments represented by the vertices connected by the edge;
• and
• generate subgraphs for vertices that include a home vertex and neighboring vertices of the home vertex representing segments of the target image;
• and
• train the autoencoder using the subgraphs as training data, the autoencoder for generating latent variables, wherein the representation of the target image is derived from the latent variables;
• and
• one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums.