Motion blur and depth of field reconstruction through temporally stable neural networks

A system, comprising:

• a memory storing a sequence of input data, the sequence including a first input data frame and a second input data frame;
• and
• at least one processor in communication with the memory and configured to:
• process an input feature vector for the first input data frame using layers of a neural network model to produce external state including a reconstructed first data frame,
• warp the external state, using difference data corresponding to changes between the first input data frame and the second input data frame, to produce warped external state, and
• process an input feature vector for the second input data frame using the layers of the neural network model to produce a reconstructed second data frame,
• wherein the input feature vector for the second input data frame includes:
• the second input data frame, the warped external state corresponding with the first input data frame, and guide data for one or more effects.

A non-transitory computer-readable media storing computer instructions for processing images with a neural network model that, when executed by a processor, cause the processor to perform the steps of:

• processing an input feature vector for a first input data frame of a sequence of input data using layers of a neural network model to produce external state including a reconstructed first data frame;
• warping the external state, using difference data corresponding to changes between the first input data frame and a second input data frame of the sequence of input data, to produce warped external state;
• and
• processing an input feature vector for the second input data frame using the layers of the neural network model to produce a reconstructed second data frame,
• wherein the input feature vector for the second input data frame includes:
• the second input data frame, the warped external state corresponding with the first input data frame, and guide data for one or more effects.