# US 10,375,406 B2 · Image encoding device and method, and image processing device and method for enabling bitstream concatenation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,406 B2 - Document ID: US10375406B2 - Status: Granted - Kind: B2 - Application number: 15/114,301 - Filed: 2015-02-24 - Published: 2017-01-12 - Granted: 2019-08-06 - Earliest claimed: 2014-03-07 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Junichi Tanaka; Akihiro Imamura; Ohji Nakagami; Takefumi Nagumo; Yuichi Araki; Yuji Fujimoto; Shinobu Hattori; Masakazu Kouno - Web page: https://patentgrep.com/patent/US10375406B2 ## CPC - H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream - H04L 65/70: Media network packetisation - H04N 19/114: Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames (H04N19/107 takes precedence) - H04N 19/126: Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers - H04N 19/184: the unit being bits, e.g. of the compressed video stream - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/463: by compressing encoding parameters before transmission - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract There is provided an image encoding device and an image encoding method, and an image processing device and an image processing method that enable easier bitstream concatenation. A structure according to the present disclosure includes: a setting unit that sets header information related to a hypothetical reference decoder in accordance with information about a position and information about reference, the information about a position and the information about reference being of the current picture of image data to be processed; and an encoding unit that encodes the image data and generates a bitstream containing the encoded data of the image data and the header information set by the setting unit. The present disclosure can be applied to image processing devices or image encoding devices, for example. ## Claims ### Claim 1 An image encoding device comprising: a setting unit configured to set, prior to concatenation of a bitstream with another bitstream, header information related to a hypothetical reference decoder, wherein the header information is set based on information about a position of a current picture of image data to be processed and information about whether the current picture is a reference picture to be referred to; and an encoding unit configured to encode the image data and generate the bitstream containing the encoded data of the image data and the header information set by the setting unit, the encoding enabling concatenation of the bitstream with the another bitstream, wherein the setting unit is further configured to set information indicating a difference between a position of an access unit at an end of the bitstream and a position of a previous non-discardable picture, based on whether or not the current picture is a first picture of a concatenating bitstream, a last picture of a bitstream to be concatenated, and the reference picture to be referred to, and wherein the setting unit and the encoding unit are each implemented via at least one processor. ### Claim 2 (depends on 1) The image encoding device according to claim 1, wherein the setting unit is further configured to set information indicating a NAL unit type. ### Claim 3 (depends on 2) The image encoding device according to claim 2, wherein the setting unit is further configured to set information indicating bitstream concatenation. ### Claim 4 (depends on 3) The image encoding device according to claim 3, wherein, when the current picture is the first picture of the concatenating bitstream, the setting unit is further configured to set the information indicating the NAL unit type at a value indicating an Instantaneous Decoding Refresh (IDR) picture, set the information indicating bitstream concatenation at “true”, and set the information indicating the difference between the position of the access unit at the end of the bitstream and the position of the previous non-discardable picture as 0. ### Claim 5 (depends on 3) The image encoding device according to claim 3, wherein, when the current picture is the last picture of the bitstream to be concatenated, the setting unit is further configured to set the information indicating the NAL unit type at a value indicating a trailing picture to be referred to, the trailing picture not being of a temporal sublayer, set the information indicating bitstream concatenation at “false”, and set the information indicating the difference between the position of the access unit at the end of the bitstream and the position of the previous non-discardable picture as 0. ### Claim 6 (depends on 3) The image encoding device according to claim 3, wherein, when the current picture is neither the first picture nor the last picture of the bitstream to be concatenated or the concatenating bitstream, but is the reference picture to be referred to, the setting unit is further configured to set the information indicating the NAL unit type at a value indicating a trailing picture to be referred to, the trailing picture not being of a temporal sublayer, set the information indicating bitstream concatenation at “false”, and set the information indicating the difference between the position of the access unit at the end of the bitstream and the position of the previous non-discardable picture as 0. ### Claim 7 (depends on 3) The image encoding device according to claim 3, wherein, when the current picture is neither the first picture nor the last picture of the bitstream to be concatenated or the concatenating bitstream, and is not the reference picture to be referred to, the setting unit is further configured to set the information indicating the NAL unit type at a value indicating a non-reference picture, the non-reference picture not being of a temporal sublayer, set the information indicating bitstream concatenation at “false”, and set the information indicating the difference between the position of the access unit at the end of the bitstream and the position of the previous non-discardable picture as 0. ### Claim 8 (depends on 1) The image encoding device according to claim 1, further comprising: a rate control unit configured to set a target code amount value in accordance with the information about the position of the current picture, information indicating a section for adjusting the hypothetical reference decoder, and information indicating a generated code amount, wherein the rate control unit is implemented via at least one processor. ### Claim 9 An image encoding method comprising: setting, prior to concatenation of a bitstream with another bitstream, header information related to a hypothetical reference decoder, wherein the header information is set based on information about a position of a current picture of image data to be processed and information about whether the current picture is a reference picture to be referred to; and encoding the image data and generating the bitstream containing the encoded data of the image data and the set header information, the encoding enabling concatenation of the bitstream with the another bitstream, wherein the setting of the header information comprises setting information indicating a difference between a position of an access unit at an end of the bitstream and a position of a previous non-discardable picture, based on whether or not the current picture is a first picture of a concatenating bitstream, a last picture of a bitstream to be concatenated, and the reference picture to be referred to. ### Claim 10 An image processing device comprising an updating unit configured to update, prior to concatenation of a bitstream with another bitstream, header information related to a hypothetical reference decoder, the header information being included in the bitstream containing encoded data generated by encoding image data and set based on information about a position of a current picture of image data to be processed and information about whether the current picture is a reference picture to be referred to, the updating enabling concatenation of the bitstream with the another bitstream, wherein the updating unit is further configured to search for a previous non-discardable picture at an end of the bitstream, and in accordance with a result of the search, update a difference between a position of an access unit at the end of the bitstream and a position of the previous non-discardable picture, based on whether or not the current picture is a first picture of a concatenating bitstream, a last picture of a bitstream to be concatenated, and the reference picture to be referred to, and wherein the updating unit is implemented via at least one processor. ### Claim 11 (depends on 10) The image processing device according to claim 10, wherein the updating unit re-encodes the bitstream to appropriately adjust a relationship between a position of a coded picture buffer at an end of the bitstream to be concatenated and a position of a coded picture buffer at a start of the concatenating bitstream. ### Claim 12 (depends on 10) The image processing device according to claim 10, wherein the updating unit updates information indicating a NAL unit type at an end of the bitstream to be concatenated with a value corresponding to a previous non-discardable picture. ### Claim 13 (depends on 10) The image processing device according to claim 10, wherein the updating unit updates information, about readout from a coded picture buffer, with a value suitable for bitstream concatenation. ### Claim 14 (depends on 10) The image processing device according to claim 10, wherein the updating unit updates information, about readout from a coded picture buffer and a decoded picture buffer at an end of the bitstream to be concatenated, with a value suitable for bitstream concatenation. ### Claim 15 (depends on 10) The image processing device according to claim 10, wherein the updating unit updates information, about readout from a coded picture buffer and a decoded picture buffer at a start of the concatenating bitstream, with a value suitable for bitstream concatenation. ### Claim 16 (depends on 15) The image processing device according to claim 15, wherein the updating unit updates information indicating a delay of readout from a coded picture buffer of an access unit at the start of the concatenating bitstream, with a value in accordance with information indicating a delay of readout from a coded picture buffer at an end of the bitstream to be concatenated. ### Claim 17 (depends on 10) The image processing device according to claim 10, further comprising a concatenating unit configured to concatenate the bitstream updated by the updating unit with the another bitstream, wherein the concatenating unit is implemented via at least one processor. ### Claim 18 An image processing method comprising updating, prior to concatenation of a bitstream with another bitstream, header information related to a hypothetical reference decoder, the header information being included in the bitstream containing encoded data generated by encoding image data and set based on information about a position of a current picture of image data to be processed and information about whether the current picture is a reference picture to be referred to, the updating enabling concatenation of the bitstream with the another bitstream, wherein the updating the header information comprises searching for a previous non-discardable picture at an end of the bitstream, and in accordance with a result of the search, updating a difference between a position of an access unit at the end of the bitstream and a position of the previous non-discardable picture, based on whether or not the current picture is a first picture of a concatenating bitstream, a last picture of a bitstream to be concatenated, and the reference picture to be referred to. ### Claim 19 (depends on 1) The image encoding device according to claim 1, wherein the setting unit is further configured to set the header information in accordance with information about whether the current picture is a first picture of a concatenating bitstream or a last picture of a bitstream to be concatenated. ### Claim 20 (depends on 1) The image encoding device according to claim 1, wherein the setting unit is further configured to determine a position of the current picture of image data to be processed; and set the header information based on the determined position of the current picture.