Method and system for 3D hair virtual try on using physics simulation

A system comprising:

• at least one processor;
• and
• at least one memory device storing computer readable instructions that, when executed by the at least one processor, cause the system to:
• process an input image using a three dimensional (3D) face detector network to determine a 3D representation of a face of a head, the input image comprising the face and the head;
• determine a 3D hairstyle model of a 3D hairstyle to be applied to the input image;
• render the 3D hairstyle from the 3D hairstyle model to define an output image, wherein to render is responsive to a physics simulation of hair movement and hair position of the 3D hairstyle to be rendered, the physics simulation responsive to the 3D representation of the face and the physics simulation modeling at least one force and deforming the 3D hairstyle in accordance with the at least one force;
• and
• provide the output image for display by a display device,
• wherein the computer readable instructions, when executed by the at least one processor, further cause the system to;
• process the input image to determine at least one of a position of the face or a rotation of the face;
• and wherein the physics simulation is responsive to at least one of the position of the face or the rotation of the face;
• and
• define a collision 3D shape representation for at least some objects of the input image, the at least some objects selected from the head, a neck and a shoulder;
• and wherein the physics simulation of the hair movement and hair position is responsive to one or more collisions of hair strands with the collision 3D shape representation.

A system comprising:

• at least one processor;
• and
• at least one memory device storing computer readable instructions that, when executed by the at least one processor, cause the system to:
• process an input image using a three dimensional (3D) face detector network to determine a 3D representation of a face of a head, the input image comprising the face and the head;
• determine a 3D hairstyle model of a 3D hairstyle to be applied to the input image;
• render the 3D hairstyle from the 3D hairstyle model to define an output image, wherein to render is responsive to a physics simulation of hair movement and hair position of the 3D hairstyle to be rendered, the physics simulation responsive to the 3D representation of the face and the physics simulation modeling at least one force and deforming the 3D hairstyle in accordance with the at least one force;
• and
• provide the output image for display by a display device,
• wherein the 3D hairstyle model models a plurality of hair strands defining the hairstyle, the model comprising an embedded frame to guide the movement of the hair strands, the frame comprising a set of guided strands distributed throughout the model, each hair strand mapped to at least one of the guided strands, each individual hair strand and each individual guided strand extending from a respective individual root of the hair outward in accordance with a shape of the hair model;
• and wherein the physics simulation determines the hair movement and the hair position of the guided strands according to the at least one force and determines hair movement and hair position of the hair strands according to the mapping to the guided strands.

A system comprising:

• at least one processor;
• and
• at least one memory device storing computer readable instructions that, when executed by the at least one processor, cause the system to:
• process an input image using a three dimensional (3D) face detector network to determine a 3D representation of a face of a head, the input image comprising the face and the head;
• determine a 3D hairstyle model of a 3D hairstyle to be applied to the input image;
• render the 3D hairstyle from the 3D hairstyle model to define an output image, wherein to render is responsive to one or more of:
• a physics simulation of hair movement and hair position of the 3D hairstyle to be rendered, the physics simulation responsive to the 3D representation of the face and the physics simulation modeling at least one force and deforming the 3D hairstyle in accordance with the at least one force;
• or
• a hair color to recolor the hair of the 3D hairstyle model, the hair color refined in accordance with an estimation of ambient light conditions;
• and
• provide the output image for display by a display device,
• wherein the computer readable instructions, when executed by the at least one processor, further cause the system to define a rendered pixel depth map for pixels rendered for the output image, initializing the depth map responsive to pixel depths for the face, a neck and at last one shoulder of the input image, and updating the depth map as pixels are rendered to the output image;
• and using the rendered pixel depth map to determine hair pixel occlusion.

A system comprising:

• at least one processor;
• and
• at least one memory device storing computer readable instructions that, when executed by the at least one processor, cause the system to:
• process an input image using a three dimensional (3D) face detector network to determine a 3D representation of a face of a head, the input image comprising the face and the head;
• determine a 3D hairstyle model of a 3D hairstyle to be applied to the input image;
• render the 3D hairstyle from the 3D hairstyle model to define an output image, wherein to render is responsive to one or more of:
• a physics simulation of hair movement and hair position of the 3D hairstyle to be rendered, the physics simulation responsive to the 3D representation of the face and the physics simulation modeling at least one force and deforming the 3D hairstyle in accordance with the at least one force;
• or
• a hair color to recolor the hair of the 3D hairstyle model, the hair color refined in accordance with an estimation of ambient light conditions;
• and
• provide the output image for display by a display device,
• wherein the computer readable instructions, when executed by the at least one processor, further cause the system to define the 3D hairstyle model as a hair cards model using 3D rectangular planes;
• and skip a rendering of a transparent hair pixel to the output image defined from the 3D hairstyle model.

A system comprising:

• at least one processor;
• and
• at least one memory device storing computer readable instructions that, when executed by the at least one processor, cause the system to:
• process an input image using a three dimensional (3D) face detector network to determine a 3D representation of a face of a head, the input image comprising the face and the head;
• determine a 3D hairstyle model of a 3D hairstyle to be applied to the input image;
• render the 3D hairstyle from the 3D hairstyle model to define an output image, wherein to render is responsive to one or more of:
• a physics simulation of hair movement and hair position of the 3D hairstyle to be rendered, the physics simulation responsive to the 3D representation of the face and the physics simulation modeling at least one force and deforming the 3D hairstyle in accordance with the at least one force;
• or
• a hair color to recolor the hair of the 3D hairstyle model, the hair color refined in accordance with an estimation of ambient light conditions;
• and
• provide the output image for display by a display device,
• wherein the computer readable instructions, when executed by the at least one processor, further cause the system to apply two or more hair colors to respective hair segments from the 3D hairstyle model, wherein the hair segments are arranged as any of horizontally, vertically, or as root recoloring.