Training of a machine learning model to generate a 3d image

An apparatus comprising:

• at least one processor;
• and
• at least one memory including computer program code;
• the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:
• in a first training process:
• receive a first plurality of two-dimensional (2D) training images;
• generate a scene representation based on the first plurality of 2D training images using a machine learning model;
• generate a first loss based on comparing the scene representation to a ground-truth scene representation;
• and
• train the machine learning model based on the first loss;
• and
• in a second training process:
• receive a first video frame including a second plurality of 2D training images;
• receive a second video frame including a third plurality of 2D training images the first frame and the second frame being captured sequentially in time;
• generate a first intermediate prediction based on the first video frame using the machine learning model;
• generate a second intermediate prediction based on the second video frame using the machine learning model;
• generate a second loss based on the first intermediate prediction and the second intermediate prediction;
• and
• train the machine learning model based on the second loss.

A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by at least one processor, are configured to cause a computing system to:

• in a first training process:
• receive a first plurality of two-dimensional (2D) training images;
• generate a scene representation based on the first plurality of 2D training images using a machine learning model;
• generate a first loss based on comparing the scene representation to a ground-truth scene representation;
• and
• train the machine learning model based on the first loss;
• and
• in a second training process:
• receive a first video frame including a second plurality of 2D training images;
• receive a second video frame including a third plurality of 2D training images the first frame and the second frame being captured sequentially in time;
• generate a first intermediate prediction based on the first video frame using the machine learning model;
• generate a second intermediate prediction based on the second video frame using the machine learning model;
• generate a second loss based on the first intermediate prediction and the second intermediate prediction;
• and
• train the machine learning model based on the second loss.