Method of creating animatable digital clone from multi-view images

A method of creating an animatable digital clone from multi-view images, comprising:

• receiving input multi-view images of an actor captured by at least two cameras;
• reconstructing 3D shape information of the actor selectively based on a probability of photo-consistency in the input multi-view images and transferring a mesh topology of a reference human body model onto a shape of the actor obtained from the reconstruction;
• generating an initial human body model of the actor via transfer of the mesh topology utilizing sectional shape information of the actor's joints by controlling a shape and a posture of the reference human body model;
• generating a genuine human body model of the actor from learning genuine behavioral characteristics of the actor by applying the initial human body model to multi-view posture learning images where performance of a predefined motion by the actor is recorded;
• and
• reconstructing information of an appearance and a motion of the actor by using the initial human body model and multi-view images of respective frames of the multi-view posture learning images as an input,
• wherein the transferring includes:
• combining the probability of photo-consistency, an anthropometric shape information and the mesh topology of the reference human body model;
• accepting reconstructed information obtained from the reconstruction when the probability is high;
• and
• accepting the anthropometric shape information when the probability is low,
• wherein each part of the reference human body model is constructed by parametric representation of sub-joint structure and local mesh topology, thereby allowing shape deformation and motion control performed in parametric control manner,
• wherein the reconstructed 3D shape of the actor is separated into at least five parts and each part is mapped to the corresponding part of the reference human body model, and assigned a motional nurbs curve representing a motion of skeletal sub-point structure and a sweep surface represented by a nurbs surface, and
• wherein, each respective sweep surface of the reference human body model is controlled and deformed parametrically via the motional nurbs curve based on posture information of the actor and via the nurbs surface based on the 3D shape information of the actor.