Real-time performance capture with on-the-fly correctives

#The present application is a non-provisional of and claims the benefit and priority under 35 U.S.C. 119(e) of U.S. Provisional Application No. 61/881,173 filed Sep. 23, 2013, entitled “REAL-TIME FACIAL ANIMATION WITH ON-THE-FLY CORRECTIVES,” the entire contents of which are incorporated herein by reference for all purposes.

#The present disclosure relates generally to performance capture, and more specifically to real-time performance capture techniques with on-the-fly correctives.

#Performance capture techniques generally rely on generic expressions or require extensive training sessions in order to capture expressions of a subject (e.g., facial expressions). Such generic expressions do not accurately reflect the nuances of the subject's expressions. In order to create subject-specific output, pre-captured expressions may be used to train a system for tracking the expressions of the subject. A subject may be required to attend a lengthy training session in order to capture specific expressions so that the system can be adequately trained to model the expressions of the subject. Furthermore, performance capture methods may need to interpolate expressions of the subject due to lack of a particular pre-processed expression being captured in the training session. These interpolated expressions oftentimes lack an accurate depiction of the subtle traits of the subject. As a result, subtle nuances of a subject's expressions are difficult to capture due to the inability of the training sessions to capture every possible expression of a user before tracking begins and also due to interpolation methods lacking the ability to provide accurate results.

#The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

#Illustrative embodiments of the present invention are described in detail below with reference to the following drawing figures:

#FIG. 1 illustrates an example of a computer system for implementing various embodiments of the present invention.

#In the following description, for the purposes of explanation, specific details are set forth in order to provide a thorough understanding of embodiments of the invention. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and description are not intended to be restrictive.

#The ensuing description provides exemplary embodiments only, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of the exemplary embodiments will provide those skilled in the art with an enabling description for implementing an exemplary embodiment. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the invention as set forth in the appended claims.

#A goal in the generation of high quality performance-driven facial animation is to capture every trait and characteristic of a subject's facial and verbal expressions, and to reproduce these expressions on a digital double or a creature. Many existing animation capture techniques rely on generic expressions that do not accurately reflect the nuances of the subject's expressions. Furthermore, creating realistic digital renderings of a subject (e.g., an actor's face) in film and game production typically involves a very complex pipeline requiring intensive manual intervention. Thus, long turn-around times may be required for generating compelling results. As a consequence of the complex pipeline and long turn-around times, high production costs may be incurred in order to generate realistic animations.