Video reenactment taking into account temporal information

A method for inserting identity information from a source image of a first subject into a destination video of a second subject different than the first subject while mimicking motion in the destination video, said method comprising the steps of:

• encoding identity information from the source image to produce an identity vector;
• encoding pose information from the destination video to produce a pose vector;
• separate from and independent of the pose information encoding step, encoding motion information from the destination video to produce a motion vector;
• and
• activating a neural network generator to accept as inputs the identity vector, the pose vector, and the motion vector;
• and to generate in response thereto a composite video comprising identity information from the source video inserted into the destination video, where the composite video has substantially the same temporal information as in the destination video.

An apparatus for inserting identity information from a source image of a first subject into a destination video of a second subject different than the first subject while mimicking motion in the destination video, said apparatus comprising:

• an identity encoder configured to encode identity information from the source image and to produce an identity vector;
• a pose encoder configured to encode pose information from the destination video, and to produce a pose vector;
• and
• a neural network generator and sequencer having two inputs:
• the identity vector and the pose vector;
• wherein
• the neural network generator and sequencer is configured to encode motion in the destination video and to generate a composite video comprising identity information from the source image inserted into the destination video, where the composite video has substantially the same temporal information as in the destination video.

A method for inserting identity information from a source image of a first subject into a destination video of a second subject different than the first subject while mimicking motion in the destination video, said method comprising the steps of:

• encoding identity information from the source image;
• using the encoded identity information from the source image to produce an identity vector;
• encoding pose information from the destination video to produce a pose vector;
• activating a neural network generator and sequencer to accept as inputs the identity vector and the pose vector, to encode motion in the destination video, and to generate a composite video comprising identify information from the source image inserted into the destination video, where the composite video has substantially the same temporal information as in the destination video.