Multi-dimensional generative framework for video generation

A non-transitory computer-readable medium having computer-executable instructions stored thereon that, upon execution, cause one or more processors to perform operations comprising:

• providing an input data to a multi-dimensionally aware generator of a multi-dimensional video generative model;
• generating the multi-dimensional video from the input data by the multi-dimensionally aware generator by:
• inverting the input data into a latent space of the multi-dimensionally aware generator;
• synthesizing content of the multi-dimensional video using an appearance component of the multi-dimensionally aware generator and corresponding camera pose and formulating an intermediate appearance code;
• developing a synthesis layer for encoding a motion component of the multi-dimensionally aware generator at a plurality of timesteps and formulating an intermediate motion code;
• introducing temporal dynamics into the intermediate appearance code and the intermediate motion code;
• and
• generating multi-dimensionally aware spatio-temporal representations of the input data.

A generator of a multi-dimensional video generative model for generating multi-dimensional video, the generator comprising:

• a control logic module to obtain an input of an input data;
• a model to generate the multi-dimensional video from the input data by:
• inverting the input data into a latent space of the multi-dimensionally aware generator;
• synthesizing content of the multi-dimensional video using an appearance component of the multi-dimensionally aware generator and corresponding camera pose and formulating an intermediate appearance code;
• developing a synthesis layer for encoding a motion component of the multi-dimensionally aware generator at a plurality of timesteps and formulating an intermediate motion code;
• introducing temporal dynamics into the intermediate appearance code and the intermediate motion code;
• and
• generating multi-dimensionally aware spatio-temporal representations of the input data.