Reducing latency in video encoding and decoding

In a computing system that implements a video decoder, a method comprising:

• receiving, at the video decoder, encoded data in a bitstream for at least some frames of a video sequence, the encoded data including:
• one or more first syntax elements that indicate a maximum size of frame memory for reordering, wherein the maximum size of frame memory for reordering is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in coded order but follow that frame in output order;
• and
• one or more second syntax elements, different than the one or more first syntax elements, that indicate a constraint on frame reordering latency, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order;
• decoding, with the video decoder, the encoded data, including:
• decoding the one or more first syntax elements;
• decoding the one or more second syntax elements;
• and
• decoding at least some of the encoded data to reconstruct one of the at least some frames of the video sequence;
• and
• outputting the reconstructed frame.

A computer system comprising a processor and memory, wherein the computer system implements a video encoder configured to perform operations comprising:

• with the video encoder, encoding at least some frames of a video sequence, thereby producing encoded data, including:
• encoding one or more first syntax elements that indicate a maximum size of frame memory for reordering, wherein the maximum size of frame memory for reordering is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in coded order but follow that frame in output order;
• and
• encoding one or more second syntax elements, different than the one or more first syntax elements, that indicate a constraint on frame reordering latency, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order;
• and
• encoding the at least some frames of the video sequence, wherein the encoding the at least some frames uses inter-frame dependencies that are consistent with the constraint on frame reordering latency;
• and
• outputting, from the video encoder, the encoded data as part of a bitstream, the encoded data including the one or more first syntax elements and the one or more second syntax elements, thereby facilitating determination of when reconstructed frames are ready for output in terms of output order.

One or more computer-readable media having stored thereon encoded data in a bitstream for at least some frames of a video sequence, the encoded data including (a) one or more first syntax elements that indicate a maximum size of frame memory for reordering, wherein the maximum size of frame memory for reordering is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in coded order but follow that frame in output order, and (b) one or more second syntax elements, different than the one or more first syntax elements, that indicate a constraint on frame reordering latency, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order, and wherein the encoded data is organized to facilitate decoding the encoded data by operations comprising, with a video decoder:

• decoding the one or more first syntax elements;
• decoding the one or more second syntax elements;
• decoding at least some of the encoded data to reconstruct one of the at least some frames of the video sequence;
• and
• outputting the reconstructed frame.