Systems and methods for encoding and playing back video at different frame rates using enhancement layers

A source encoder, comprising:

• a processor configured by a source encoder application to:
• obtain video comprising a merged plurality of access units with temporal scalability, each access unit in the merged plurality of access units associated with a particular layer of a plurality of temporal layers based on a temporal identifier of the access unit that identifies the particular layer, wherein the temporal layers include a base layer and one or more enhancement layer;
• obtain an ordered sequence of access units ordered based on a decoding order for decoding the video and generating metadata specifying an ordered sequence of temporal identifiers ordered based on, for each access unit in the ordered sequence of access units, identifying and retaining the temporal identifier associated with the access unit;
• separate, based on the temporal identifier of each access unit in the ordered sequence of access units, the ordered sequence of access units into (1) a base layer of access units sequenced in decode order to provide a base frame rate and (2) at least one enhancement layer with additional access units ordered in decode order to provide an enhanced frame rate, wherein at least some of the access units in the enhancement layer are encoded based upon dependencies to access units in the base layer, and wherein access units in the base layer are encoded without dependencies upon access units in the enhancement layer;
• storing the base layer in a first container file;
• and
• storing the enhancement layer, including the generated metadata specifying the ordered sequence of temporal identifiers, in a second container file separate from the first container file, wherein the metadata is used to recombine in decoding order the access units stored in the second container file with access units stored in the first container file for playback at the enhanced frame rate.

A playback device configured to playback video comprising:

• a processor configured to communicate with a memory, where the memory contains a client application;
• wherein the client application configures the processor to:
• obtain a first container file comprising a base layer of access units ordered in decode order for playback at a base frame rate;
• obtain, for a device capable of playing back an enhancement layer, a second container file comprising (1) at least one enhancement layer with additional access units ordered in decode order for playback at an enhanced frame rate and (2) metadata specifying an ordered sequence of temporal identifiers for combining, in decode order, access units from the base layer with access units from the at least one enhancement layer for playback at the enhanced frame rate, wherein each access unit includes a temporal identifier identifying a particular layer associated with the access unit;
• combine the base layer of access units with the at least one enhancement layer based on the metadata obtained from the second container file to obtain a combined sequence of access units ordered in a decoding order at the enhanced frame rate;
• wherein at least some of the access units in the at least enhancement layer stored in the second container file are decoded based upon dependencies to access units in the base layer stored in the first container file, and wherein access units in the base layer that are stored in the first container file are decoded without dependencies upon access units in the enhancement layer.