Deep Multi-Scale Video Prediction

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• receive a plurality of input frames of a video sequence associated with a time t;
• predict, using a convolutional network, one or more future frames of the video sequence from the plurality of input frames, wherein the convolutional network is trained with randomly selected temporal sequences of a n×m grid of pixels from the plurality of input frames exhibiting a threshold of optical flow;
• and
• output a first future frame of the video sequence associated with a time t+1 as predicted by the convolutional network.

A system comprising:

• one or more processors;
• and one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
• receive a plurality of input frames of a video sequence associated with a time t;
• predict, using a convolutional network, one or more future frames of the video sequence from the plurality of input frames, wherein the convolutional network is trained with randomly selected temporal sequences of a n×m grid of pixels from the plurality of input frames exhibiting a threshold of optical flow;
• and
• output a first future frame of the video sequence associated with a time t+1 as predicted by the convolutional network.