Neural frame rate upsampling via learned alpha

A computing device, comprising:

• a memory comprising one more storage devices;
• and
• one or more processors coupled to the memory, the one or more processors operable to execute instructions stored in the memory to, for a rendered image sequence:
• create a first interpolated optical flow frame based at least on a first preceding frame or a first following frame and optical flow from the first preceding frame or first following frame;
• create a first interpolated motion vector frame based, at least in part, on a second preceding frame or a second following frame and motion vectors from the second preceding frame or the second following frame;
• provide the first interpolated optical flow frame and the first interpolated motion vector frame to a neural network trained to predict blending parameters for blending each of the first interpolated optical flow frame and the first interpolated motion vector frame to generate an interpolated output frame, and generate and output predicted blending parameters via the neural network;
• and
• generate the interpolated output frame by applying the predicted blending parameters to blend the first interpolated optical flow frame and the first interpolated motion vector frame.

A method of training a neural network, comprising:

• receiving an input tensor in an input layer of a neural network, the input tensor representing one or more characteristics of an image;
• providing an output tensor to an output layer of the neural network, the output tensor representing:
• one or more coefficients predicting blending parameters to be used in blending at least a first interpolated optical flow frame based at least on a first preceding frame or first following frame and optical flow from the first preceding frame or first following frame, and a first interpolated motion vector frame based at least on a second preceding frame or second following frame and motion vectors from the second preceding frame or second following frame, the blending parameters generated at least in part by providing the first interpolated optical flow frame and the first interpolated motion vector frame the neural network as an input tensor;
• training the neural network to predict the provided output tensor based on the received input tensor by using backpropagation to adjust a weight of one or more activation functions linking one or more nodes of one or more layers of the neural network.