Advanced Systems and Methods for Automatically Generating an Animatable Object from Various Types of User Input

A system for automatically generating an animatable object, the system comprising:

• a processor;
• and
• a memory for storing executable instructions, the processor executing the instructions to:
• receive a first input from a first device and a second input from the first device or from a second device, each of the first and second input comprising a point cloud of an image associated with the face or head of a user, the first input being associated with at least one sensor on the first device and the second input being associated with the at least one sensor on the first device, at least one different sensor on the first device, or at least one different sensor on at least one other device;
• preprocess the first and second input, comprising:
• determining orientation of the point clouds for the first input and the second input, the orientation of the point clouds of the image in space being unknown based on the first and second inputs as the at least one sensor is in an unknown position regarding location with respect to the face the user and the determining the orientation for the first input and the second input using photogrammetry includes quantifying each of the point cloud images from the first and second inputs regarding location with respect to the face the user;
• register by means of rigid or non-rigid registration the point clouds from the first and second inputs to align the preprocessed point clouds to a common reference;
• and
• after registering:
• automatically detect one or more features of the registered point clouds;
• deform a template geometry based on the one or more detected features to automatically generate a custom geometry;
• determine a texture of the received first and second inputs and transferring the texture to the custom geometry;
• deform a template control structure based on the one or more detected features to automatically generate a custom control structure;
• and
• generate an animatable object having the custom geometry, the transferred texture, and the custom control structure.

A system for creating a customized animatable 3D model from a combination of input devices and sensors, the system comprising:

• means for receiving a first input from a first device and a second input from the first device or from a second device, each of the first input and second input comprising a point cloud of an image associated with the face or head of a user, the first input being associated with at least one sensor on the first device and the second input being associated with the at least one sensor on the first device, at least one different sensor on the first device, or at least one different sensor on at least one other device;
• means for preprocessing the first input and second input, the preprocessing comprising:
• determining orientation of the point clouds for the first input and the second input, the orientation of the point clouds of the image in space being unknown based on the first and second inputs as the at least one sensor is in an unknown position regarding location with respect to the face the user and the determining the orientation for the first input and the second input using photogrammetry includes quantifying each of the point cloud images from the first and second inputs regarding location with respect to the face the user;
• means for registering the point clouds from the first and second inputs to align the preprocessed point clouds to a common reference;
• and
• after registering:
• means for automatically detecting one or more features of the registered point clouds;
• means for deforming a template geometry based on the one or more detected features to automatically generate a custom geometry;
• means for determining a texture of the received first and second inputs and transferring the texture to the custom geometry;
• means for deforming a template control structure based on the one or more detected features to automatically generate a custom control structure;
• and
• means for generating an animatable object having the custom geometry, the transferred texture, and the custom control structure.