Tunable Hybrid Neural Video Representations

A system comprising:

• a neural network-based video decoder configured to:
• receive a latent feature corresponding to a compressed version of a first video frame of a first plurality of a plurality of video frames included in a video sequence, the latent feature being a combination of a frame-specific embedding of the first video frame with one or more group-of-pictures (GOP) features of the first plurality of the plurality of video frames;
• and
• decode the latent feature to provide an uncompressed video frame corresponding to the first video frame.

A method for execution by a tunable neural network-based video encoder, the method comprising:

• receiving a video sequence including a plurality of video frames;
• generating a frame-specific embedding of a first video frame of the plurality of video frames;
• identifying one or more group-of-pictures (GOP) features of a first plurality of the plurality of video frames, the first plurality of the plurality of video frames including the first video frame;
• and
• combining the frame-specific embedding of the first video frame and the one or more GOP features of the first plurality of the plurality of video frames to provide a latent feature corresponding to a compressed version of the first video frame.