Retargeting skeleton motion sequences through cycle consistency adversarial training of a motion synthesis neural network with a forward kinematics layer

A system for training motion synthesis neural networks to generate retargeted skeleton motion sequences that reflect motion sequences of initial skeletons comprising:

• at least one processor;
• at least one non-transitory computer memory comprising a motion synthesis neural network that includes an encoder recurrent neural network, a decoder recurrent neural network, and a forward kinematics layer;
• and
• instructions that, when executed by at least one processor, cause the system to:
• provide training input joint features for joints of a training initial skeleton to the motion synthesis neural network, wherein the training input joint features correspond to an initial time of a training motion sequence;
• utilize the encoder recurrent neural network and the decoder recurrent neural network to generate predicted joint rotations for joints of a training target skeleton for an initial time of a training target motion sequence based on the training input joint features;
• utilize the forward kinematics layer to generate predicted joint features for joints of the training target skeleton for the initial time of the training target motion sequence based on the predicted joint rotations;
• and
• train the motion synthesis neural network to generate target skeleton motion sequences from initial skeleton motion sequences based on the predicted joint features for the joints of the training target skeleton.

A method for training and applying motion synthesis neural networks comprising:

• performing a step for training a motion synthesis neural network to generate training target motion sequences for training target skeletons from training motion sequences of training initial skeletons;
• performing a step for generating a target motion sequence for joints of a target skeleton based on an initial motion sequence for joints of an initial skeleton using the trained motion synthesis neural network;
• and
• rendering an animated object performing the target motion sequence for the joints of the target skeleton.