Video codec buffer quantity reduction

A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• determining, from a plurality of prediction modes that can be used to determine historical data for a frame data from a frame in a video sequence, a prediction mode for data that represents the frame data in the frame;
• in response to determining the prediction mode, selecting, using the prediction mode, one or more buffers from a plurality of buffers, wherein:
• each buffer from the plurality of buffers is for a corresponding prediction mode from the plurality of prediction modes,
• a first quantity of buffers in the plurality of buffers maintained to be less than a second quantity of prediction modes in the plurality of prediction modes,
• a first set of prediction modes among the plurality of prediction modes (i) corresponds to the plurality of buffers and (ii) comprises the prediction mode, and
• a second set of prediction modes among the plurality of prediction modes does not correspond to the plurality of buffers;
• retrieving, from each of the one or more buffers, historical data for the frame data;
• and
• in response to retrieving the historical data, generating, using the historical data, updated data for the frame data in the frame of the video sequence.

A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

• determining, from a plurality of prediction modes that can be used to determine historical data for a frame data from a frame in a video sequence, a prediction mode for data that represents the frame data in the frame;
• in response to determining the prediction mode, selecting, using the prediction mode, one or more buffers from a plurality of buffers, wherein:
• each buffer from the plurality of buffers is for a corresponding prediction mode from the plurality of prediction modes,
• a first quantity of buffers in the plurality of buffers maintained to be less than a second quantity of prediction modes in the plurality of prediction modes,
• a first set of prediction modes among the plurality of prediction modes (i) corresponds to the plurality of buffers and (ii) comprises the prediction mode, and
• a second set of prediction modes among the plurality of prediction modes does not correspond to the plurality of buffers;
• retrieving, from each of the one or more buffers, historical data for the frame data;
• and
• in response to retrieving the historical data, generating, using the historical data, updated data for the frame data in the frame of the video sequence.