Systems and methods for motion estimation and view prediction

A method of view prediction, comprising:

• receiving at least one of a target time or a viewing direction associated with a scene, wherein the scene includes multiple points associated with one or more objects in the scene;
• predicting, using one or more machine-learning (ML) models, image properties of the multiple points at the target time or in the viewing direction, wherein the one or more ML models are trained to learn respective motions of the multiple points and a volumetric representation of the scene from a set of training images that depicts the scene over a time period, the one or more ML models further trained to predict the image properties of the multiple points based on the respective motions of the multiple points and the volumetric representation of the scene;
• and
• generating an image depicting the scene at the target time or in the viewing direction based on the image properties of the multiple points predicted by the one or more ML models.

A method for training machine-learning (ML) models, the method comprises:

• extracting, using present parameters of a first ML model, a plurality of features from a set of training images associated with a scene, wherein the scene includes multiple points associated with one or more objects in the scene, and wherein the plurality of features indicates respective motions of the multiple points from a source time to a target time;
• determining, using present parameters of a second ML model, a motion field that indicates respective locations of the multiple points in the scene at the target time, wherein the motion field is determined based on the plurality of features extracted by the first ML model;
• predicting, using present parameters of a third ML model, respective image properties of the multiple points at the target time or in a viewing direction based on the respective locations of the multiple points indicated by the motion field;
• generating an output image based at least on the predicted image properties of the multiple points;
• and
• adjusting the respective present parameters of the first ML model, the second ML model, and the third ML model based at least on a difference between the output image and a ground truth image.