# US 2018/0324458 A1 · Decoded picture buffer management > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0324458 A1 - Document ID: US20180324458A1 - Status: Published application - Kind: A1 - Application number: 16/032,637 - Filed: 2018-07-11 - Published: 2018-11-08 - Earliest claimed: 2011-09-23 - Assignees: Velos Media, LLC - Applicants: Velos Media, LLC - Inventors: Ye-Kui WANG; Ying CHEN - Web page: https://patentgrep.com/patent/US20180324458A1 ## CPC - H04N 19/58: Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one (H04N19/23 takes precedence) - H04N 7/12: Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal (H04N7/24 takes precedence) - H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction - H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/463: by compressing encoding parameters before transmission - H04N 19/50: using predictive coding (H04N19/61 takes precedence) - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Techniques are described related to output and removal of decoded pictures from a decoded picture buffer (DPB). The example techniques may remove a decoded picture from the DPB prior to coding a current picture. For instance, the example techniques may remove the decoded picture if that decoded picture is not identified in the reference picture set of the current picture. ## Claims ### Claim 1 A method for coding video data, the method comprising: coding information indicative of reference pictures that belong to a reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting a current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; constructing a plurality of reference picture subsets that each identifies zero or more of the reference pictures of the reference picture set, wherein constructing the plurality of reference picture subsets comprises: constructing a first reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and prior to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are prior to the current picture in output order and cannot be used for inter-predicting the current picture; constructing a second reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and subsequent to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are subsequent to the current picture in output order and cannot be used for inter-predicting the current picture; and constructing a third reference picture subset that identifies long-term reference pictures that are prior to the current picture in decoding order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes long-term reference pictures that are prior to the current picture in decoding order and that cannot be used for inter-predicting the current picture; after construction of the first, second, and third reference picture subsets, determining which reference pictures are usable for inter-prediction of the current picture based on which pictures are identified in the first, second, and third reference picture subsets; subsequent to determining which reference pictures are usable for inter-prediction of the current picture, constructing an initial reference picture list and a second reference picture list based on reference pictures in the first, second, and third reference picture subsets that are determined to be usable for inter-prediction of the current picture; and based on reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: identifying a reference picture, wherein the reference picture is in at least one of the first, second, or third reference picture subsets; adding the identified reference picture in a current entry of the initial reference picture list or in a current entry of the second reference picture list to construct a modified reference picture list or a second modified reference picture list respectively; and coding the current picture based on the modified reference picture list or on the second modified reference picture list. ### Claim 2 (depends on 1) The method of claim 1, wherein identifying the reference picture comprises: determining an index into at least one of the constructed reference picture subsets; and determining the reference picture identified at an entry of the at least one of the constructed reference picture subsets based on the determined index. ### Claim 3 (depends on 2) The method of claim 2, wherein determining the index comprises: coding a first syntax element to identify the at least one of the constructed reference picture subsets from which the reference picture is identified; and coding a second syntax element that indicates the index into the at least one of the constructed reference picture subsets. ### Claim 4 (depends on 1) The method of claim 1, further comprising: based on the reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: moving reference pictures identified in the initial reference picture list or in the second reference picture list in entries following the current entry to a next entry to construct the modified reference picture list or the second modified reference picture list. ### Claim 5 (depends on 1) The method of claim 1, further comprising: coding a syntax element that indicates that the reference picture list modification is needed. ### Claim 6 (depends on 1) The method of claim 1, wherein coding comprises decoding information indicative of reference pictures that belong to the reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting the current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; and wherein coding comprises decoding the current picture based on the modified reference picture list. ### Claim 7 (depends on 1) The method of claim 1, wherein coding comprises encoding information indicative of reference pictures that belong to the reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting the current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; and wherein coding comprises encoding the current picture based on the modified reference picture list. ### Claim 8 A device for coding video data, the device comprising: memory configured to stored reference pictures; and a video coder configured to: code information indicative of reference pictures stored in the memory that belong to a reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting a current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; construct a plurality of reference picture subsets that each identifies zero or more of the reference pictures of the reference picture set, wherein to construct the plurality of reference picture subsets, the video coder is configured to: construct a first reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and prior to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are prior to the current picture in output order and cannot be used for inter-predicting the current picture; construct a second reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and subsequent to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are subsequent to the current picture in output order and cannot be used for inter-predicting the current picture; and construct a third reference picture subset that identifies long-term reference pictures that are prior to the current picture in decoding order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes long-term reference pictures that are prior to the current picture in decoding order and that cannot be used for inter-predicting the current picture; after construction of the first, second, and third reference picture subsets, determine which reference pictures are usable for inter-prediction of the current picture based on which pictures are identified in the first, second, and third reference picture subsets; subsequent to determining which reference pictures are usable for inter-prediction of the current picture, construct an initial reference picture list and a second reference picture list based on reference pictures in the first, second, and third reference picture subsets that are determined to be usable for inter-prediction of the current picture; and based on reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: identify a reference picture, wherein the reference picture is in at least one of the first, second, or third reference picture subsets; add the identified reference picture in a current entry of the initial reference picture list or in a current entry of the second reference picture list to construct a modified reference picture list or a second modified reference picture list respectively; and code the current picture based on the modified reference picture list or on the second modified reference picture list. ### Claim 9 (depends on 8) The device of claim 8, wherein to identify the reference picture, the video coder is configured to: determine an index into at least one of the constructed reference picture subsets; and determine the reference picture identified at an entry of the at least one of the constructed reference picture subsets based on the determined index. ### Claim 10 (depends on 9) The device of claim 9, wherein to determine the index, the video coder is configured to: code a first syntax element to identify the at least one of the constructed reference picture subsets from which the reference picture is identified; and code a second syntax element that indicates the index into the at least one of the constructed reference picture subsets. ### Claim 11 (depends on 8) The device of claim 8, wherein the video coder is configured to: based on the reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: move reference pictures identified in the initial reference picture list or in the second reference picture list in entries following the current entry to a next entry to construct the modified reference picture list or the second modified reference picture list. ### Claim 12 (depends on 8) The device of claim 8, wherein the video coder is configured to: code a syntax element that indicates that the reference picture list modification is needed. ### Claim 13 (depends on 8) The device of claim 8, wherein the video coder comprises a video decoder, and wherein the video decoder is configured to: decode information indicative of reference pictures that belong to the reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting the current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; construct the first, second, and third reference picture subsets; after construction of the first, second, and third reference picture subsets, determine which reference pictures are usable for inter-prediction of the current picture based on which pictures are identified in the first, second, and third reference picture subsets; subsequent to determining which reference pictures are usable for inter-prediction of the current picture, construct the initial reference picture list and the second reference picture list based on the reference pictures in the first, second, and third reference picture subsets that are determined to be usable for inter-prediction of the current picture; and based on the reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: identify the reference picture; add the identified reference picture in the current entry of the initial reference picture list or in the current entry of the second reference picture list to construct the modified reference picture list or the second modified reference picture list respectively; and decode the current picture based on the modified reference picture list or on the second modified reference picture list. ### Claim 14 (depends on 13) The device of claim 13, further comprising a display configured to display the decoded current picture and one or more of the reference pictures. ### Claim 15 (depends on 8) The device of claim 8, wherein the video coder comprises a video encoder, and wherein the video encoder is configured to: encode information indicative of reference pictures that belong to the reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting the current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; construct the first, second, and third reference picture subsets; after construction of the first, second, and third reference picture subsets, determine which reference pictures are usable for inter-prediction of the current picture based on which pictures are identified in the first, second, and third reference picture subsets; subsequent to determining which reference pictures are usable for inter-prediction of the current picture, construct the initial reference picture list and the second reference picture list based on the reference pictures in the first, second, and third reference picture subsets that are determined to be usable for inter-prediction of the current picture; and based on the reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: identify the reference picture; add the identified reference picture in the current entry of the initial reference picture list or the current entry of the second reference picture list to construct the modified reference picture list or the second modified reference picture list respectively; and encode the current picture based on the modified reference picture list or on the second modified reference picture list. ### Claim 16 (depends on 15) The device of claim 15, further comprising a camera configured to capture the current picture that is encoded and the reference pictures. ### Claim 17 (depends on 8) The device of claim 8, wherein the device comprises one or more of: a wireless communication device comprising at least one of a display configured to display the current picture or a camera configured to capture the current picture; a microprocessor; or an integrated circuit. ### Claim 18 A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a device for coding video data to: code information indicative of reference pictures that belong to a reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting a current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; construct a plurality of reference picture subsets that each identifies zero or more of the reference pictures of the reference picture set, wherein the instructions that cause the processor to construct the plurality of reference picture subsets comprise instructions that cause the processor to: construct a first reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and prior to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are prior to the current picture in output order and cannot be used for inter-predicting the current picture; construct a second reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and subsequent to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are subsequent to the current picture in output order and cannot be used for inter-predicting the current picture; and construct a third reference picture subset that identifies long-term reference pictures that are prior to the current picture in decoding order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes long-term reference pictures that are prior to the current picture in decoding order and that cannot be used for inter-predicting the current picture; after construction of the first, second, and third reference picture subsets, determine which reference pictures are usable for inter-prediction of the current picture based on which pictures are identified in the first, second, and third reference picture subsets; subsequent to determining which reference pictures are usable for inter-prediction of the current picture, construct an initial reference picture list and a second reference picture list based on reference pictures the first, second, and third reference picture subsets that are determined to be usable for inter-prediction of the current picture; and based on the reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: identify a reference picture, wherein the reference picture is in at least one of the first, second, or third reference picture subsets; add the identified reference picture in a current entry of the initial reference picture list or in a current entry of the second reference picture list to construct a modified reference picture list or the second modified reference picture list respectively; and code the current picture based on the modified reference picture list or on the second modified reference picture list. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the instructions that cause the processor to identify the reference picture comprise instructions that cause the processor to: determine an index into at least one of the constructed reference picture subsets; and determine the reference picture identified at an entry of the at least one of the constructed reference picture subsets based on the determined index. ### Claim 20 (depends on 19) The non-transitory computer-readable storage medium of claim 19, wherein the instruction that cause the processor to determine the index comprise instructions that cause the processor to: code a first syntax element to identify the at least one of the constructed reference picture subsets from which the reference picture is identified; and code a second syntax element that indicates the index into the at least one of the constructed reference picture subsets. ### Claim 21 (depends on 18) The non-transitory computer-readable storage medium of claim 18, further comprising instructions that cause the processor to: based on the reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list: move reference pictures identified in the initial reference picture list or the second reference picture list in entries following the current entry to a next entry to construct the modified reference picture list or the second modified reference picture list. ### Claim 22 (depends on 18) The non-transitory computer-readable storage medium of claim 18, further comprising instructions that cause the processor to: code a syntax element that indicates that the reference picture list modification is needed. ### Claim 23 A device for coding video data, the device comprising: means for coding information indicative of reference pictures that belong to a reference picture set, wherein the reference picture set identifies the reference pictures that can potentially be used for inter-predicting a current picture and can potentially be used for inter-predicting one or more pictures following the current picture in decoding order; means for constructing a plurality of reference picture subsets that each identifies zero or more of the reference pictures of the reference picture set, wherein the means for constructing the plurality of reference picture subsets comprises: means for constructing a first reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and prior to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are prior to the current picture in output order and cannot be used for inter-predicting the current picture; means for constructing a second reference picture subset that identifies short-term reference pictures that are prior to the current picture in decoding order and subsequent to the current picture in output order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes short-term reference pictures that are subsequent to the current picture in output order and cannot be used for inter-predicting the current picture; and means for constructing a third reference picture subset that identifies long-term reference pictures that are prior to the current picture in decoding order and that can potentially be used for inter-predicting the current picture and one or more of the one or more pictures following the current picture in decoding order, and excludes long-term reference pictures that are prior to the current picture in decoding order and that cannot be used for inter-predicting the current picture; means for determining which reference pictures are usable for inter-prediction of the current picture based on which pictures are identified in the first, second, and third reference picture subsets after construction of the first, second, and third reference picture subsets; means for constructing, subsequent to determining which reference pictures are usable for inter-prediction of the current picture, an initial reference picture list and a second reference picture list based on reference pictures in the first, second, and third reference picture subsets that are determined to be usable for inter-prediction of the current picture; means for identifying a reference picture, wherein the reference picture is in at least one of the first, second, or third reference picture subsets based on reference picture list modification of the initial reference picture list or of the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list; means for adding the identified reference picture in a current entry of the initial reference picture list or in a current entry of the second reference picture list to construct a modified reference picture list or a second modified reference picture list respectively based on reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list; and means for coding the current picture based on the modified reference picture list or on the second modified reference picture list based on reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list. ### Claim 24 (depends on 23) The device of claim 23, wherein means for identifying the reference picture comprises: means for determining an index into at least one of the constructed reference picture subsets; and means for determining the reference picture identified at an entry of the at least one of the constructed reference picture subsets based on the determined index. ### Claim 25 (depends on 24) The device of claim 24, wherein the means for determining the index comprises: means for coding a first syntax element to identify the at least one of the constructed reference picture subsets from which the reference picture is identified; and means for coding a second syntax element that indicates the index into the at least one of the constructed reference picture subsets. ### Claim 26 (depends on 23) The device of claim 23, further comprising: means for moving reference pictures identified in the initial reference picture list or the second reference picture list in entries following the current entry to a next entry to construct the modified reference picture list or the second modified reference picture list based on the reference picture list modification of the initial reference picture list or the second reference picture list being needed after the construction of the initial reference picture list and the second reference picture list. ### Claim 27 (depends on 23) The device of claim 23, further comprising: means for coding a syntax element that indicates that the reference picture list modification is needed.