System and method for artificial neural-network based animation with three-dimensional rendering

A system for animating an image of an object, the system comprising:

• a memory;
• and
• a processor configured to:
• receive a first image, depicting a puppet object;
• sample an input video, depicting a driver object, to obtain a second image;
• extract a plurality of three dimensional (3D) features from the first image and from the second image;
• obtain, by a first machine-learning (ML) model, a value of a first 3D identity-invariant feature of the puppet object, and a value of a second 3D identity-invariant feature, from the second image;
• calculate, by a second ML model, a mixing function, based on the first 3D identity-invariant feature and the second 3D identity-invariant feature;
• apply the mixing function on the plurality of 3D features;
• generate by a third ML model, a rendered image, based on the plurality of 3D features, the first 3D identity-invariant feature and the second 3D identity-invariant feature;
• and
• generate by a fourth ML model an output image based on the rendered image, the plurality of 3D features, the first image and the second image, wherein the output image depicts a target object comprising at least one 3D identity-invariant feature of the driver object.

A method of animating an image of an object, the method comprising:

• determining a plurality of three dimensional (3D) features from a first image, depicting a first object and from a second image, wherein the second image is sampled from an input video depicting a second object;
• applying a mixing function on the plurality of 3D features, wherein the mixing function is calculated, based on a first 3D identity-invariant feature of the first object and a second 3D identity-invariant feature from the second image;
• generating a created image, based on the plurality of 3D features, the first 3D identity-invariant feature and the second 3D identity-invariant feature;
• and
• generating an output image depicting a target object comprising at least one 3D identity-invariant feature of the second object, wherein the output image is generated based on the plurality of 3D features, the created image, the first image and the second image.