Learning hybrid (surface-based and volume-based) shape representation

A computer-implemented method of training an atlas-based surface neural network for generating a three-dimensional surface from an input image comprising:

• receiving a set of input images;
• for each input image in the set of input images:
• encoding the input image to generate a multi-dimensional vector representing the input image;
• generating, using an implicit function neural network, an implicit function that, given a random point, provides a value indicating whether the random point is on a surface of a three-dimensional representation of the input image;
• generating, using the atlas-based surface neural network, a three-dimensional surface representing the input image;
• calculating a loss using the random point, the implicit function, and the three-dimensional surface;
• and
• adjusting parameters of the atlas-based surface neural network based on the loss.

A system, comprising:

• one or more processors;
• and
• a memory having stored thereon instructions that, when executed by the one or more processors, cause the one or more processors to:
• receive an input comprising a representation of an object;
• generate a multi-dimensional vector for the input;
• provide the multi-dimensional vector for the input to a neural network, wherein the neural network was trained to predict a three-dimensional surface representing the input based on the multi-dimensional vector such that the three-dimensional surface corresponds to a second three-dimensional surface of the object calculated using an implicit function for representing the input;
• and
• predict, using the neural network and based upon the multi-dimensional vector, the three-dimensional surface representing the input, the three-dimensional surface comprising a three-dimensional representation of the object.