Projecting augmentation images onto moving objects

A projection system for augmenting an appearance of a moving actor comprising:

• a lighting source that emits non-visible wavelengths of light onto the moving actor;
• a camera that captures an orientation image corresponding to reflection of the non-visible wavelengths off of the moving actor;
• a computer in electrical communication with the camera, wherein the computer performs the following operations:
• determine a blend of component attributes to define visual characteristics of the moving actor;
• modify an input image based on the orientation image to generate a modified input image;
• determine a present location of a plurality of facial landmarks on the actor based on the orientation image;
• predict a future location of the plurality of facial landmarks;
• deform a mesh model corresponding to the actor based on the predicted future location of the landmarks;
• overlay the modified input image over the deformed mesh model to generate an output image;
• and
• a projector in electrical communication with the computer, wherein the projector emits visible wavelengths of light corresponding to the output image onto the moving actor.

A visual augmentation system comprising:

• a camera;
• a projector;
• and
• a processor electrically coupled to the camera and the projector, the processor configured to:
• receive, from the camera, an image of a human face based on reflected infrared light;
• determine a weighted blend of component attributes to approximate the human face based on the received image;
• modify a texture based on the weighted blend of component attributes;
• detect a location of at least one facial feature the image of the human face;
• predict a future location of the at least one facial feature;
• generate a deformed model of the human face based on the future location of the at least one facial feature;
• apply the texture to the deformed model to generate a projectable image;
• and
• transmit for projection by the projector, the projectable image.