Methods and systems for generating an animated 3D model based on a 2D image

A system comprising:

• a memory storing instructions;
• and
• a processor communicatively coupled to the memory and configured to execute the instructions to:
• access a two-dimensional (2D) image captured by a capture device and depicting a first subject of a particular subject type;
• generate a custom three-dimensional (3D) model of the first subject, the generating comprising:
• accessing a machine learning model associated with the particular subject type,
• semantically segmenting, using the machine learning model, the 2D image to distinguish a first component associated with the first subject from one or more additional components associated with the first subject,
• determining a parameter based on a characteristic of the distinguished first component of the semantically segmented 2D,
• applying the parameter to a parametric 3D model associated with the particular subject type to generate a custom mesh for the custom 3D model, and
• applying a custom texture based on the 2D image to the custom mesh;
• access a motion capture video depicting motion performed by a second subject of the particular subject type;
• and
• animate, based on the motion capture video, the custom 3D model of the first subject to cause the custom 3D model to perform the motion performed by the second subject.

A non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to:

• access a two-dimensional (2D) image captured by a capture device and depicting a first subject of a particular subject type;
• generate a custom three-dimensional (3D) model of the first subject, the generating comprising:
• accessing a machine learning model associated with the particular subject type,
• semantically segmenting, using the machine learning model, the 2D image to distinguish a first component associated with the first subject from one or more additional components associated with the first subject,
• determining a parameter based on a characteristic of the distinguished first component of the semantically segmented 2D image,
• applying the parameter to a parametric 3D model associated with the particular subject type to generate a custom mesh for the custom 3D model, and
• applying a custom texture based on the 2D image to the custom mesh;
• access a motion capture video depicting motion performed by a second subject of the particular subject type;
• and
• animate, based on the motion capture video, the custom 3D model of the first subject to cause the custom 3D model to perform the motion performed by the second subject.