# US 9,883,207 B2 · Methods and apparatus for adaptive coupled pre-processing and post-processing filters for video encoding and decoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,207 B2 - Document ID: US9883207B2 - Status: Granted - Kind: B2 - Application number: 13/518,942 - Filed: 2010-12-02 - Published: 2012-11-08 - Granted: 2018-01-30 - Earliest claimed: 2009-12-31 - Assignees: Thomson Licensing Dtv - Applicants: Kiran Misra; Joel Sole; Peng Yin; Xiaoan Lu; Yunfei Zheng; Qian Xu - Inventors: Kiran Misra; Joel Sole; Peng Yin; Xiaoan Lu; Yunfei Zheng; Qian Xu - Web page: https://patentgrep.com/patent/US9883207B2 ## CPC - H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness - H04N 19/102: characterised by the element, parameter or selection affected or controlled by the adaptive coding - H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh - H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) - H04N 19/186: the unit being a colour or a chrominance component - H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) - H04N 19/44: Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder - H04N 19/82: involving filtering within a prediction loop - H04N 19/85: using pre-processing or post-processing specially adapted for video compression - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods and apparatus are provided for adaptive coupled pre-processing and post-processing filters for video encoding and decoding. The apparatus and method encode input data for a picture into a resultant bitstream, wherein said video encoder comprises a pre-filter and a post-filter coupled to the pre-filter, wherein said pre-filter filters the input data for the picture and the post-filter filters in-loop reconstructed data for the picture. ## Claims ### Claim 1 An apparatus, comprising: a video encoder for encoding input data for a picture into a resultant bitstream, wherein said video encoder comprises a pre-filter and a post-filter coupled to the pre-filter, wherein said pre-filter filters the input data for the picture and the post-filter filters in-loop reconstructed data for the picture, wherein at least one of filter coefficients and filter parameters for at least one of the pre-filter and the post-filter are selected responsive to at least one of a resolution, a local gradient, and a gradient direction, wherein gradient is a difference between pixels being pre-filtered, and adaptation is made discrete by partitioning a gradient strength, gradient direction and gradient variance into disjoint ranges and applying fixed filters for different ranges, and wherein a Lagrangian parameter for luminance and chrominance in the video encoder's rate-distortion optimizer is a function of the coding efficiency of the pre-filter and the post-filter; and wherein the pre-filter and post-filter are implemented with a lifting scheme and adaptation is embedded therein by changing values of a diagonal matrix, obtained by performing singular value decomposition on a fixed pre-filter transform matrix, such that values of the diagonal matrix can be reversed at a decoder. ### Claim 2 In a video encoder, a method, comprising: encoding input data for a picture into a resultant bitstream, wherein said video encoder comprises a pre-filter and a post-filter coupled to the pre-filter, wherein said pre-filter filters the input data for the picture and the post-filter filters in-loop reconstructed data for the picture, wherein at least one of filter coefficients and filter parameters for at least one of the pre-filter and the post-filter are selected responsive to at least one of a resolution, a local gradient, and a gradient direction, wherein gradient is a difference between pixels being pre-filtered, and adaptation is made discrete by partitioning a gradient strength, gradient direction and gradient variance into disjoint ranges and applying fixed filters for different ranges, and wherein a Lagrangian parameter for luminance and chrominance in the video encoder's rate-distortion optimizer is a function of the coding efficiency of the pre-filter and the post-filter; and wherein the pre-filter and post-filter are implemented with a lifting scheme and adaptation is embedded therein by changing values of a diagonal matrix, obtained by performing singular value decomposition on a fixed pre-filter transform matrix, such that values of the diagonal matrix can be reversed at a decoder. ### Claim 3 (depends on 2) The method of claim 2, wherein filter coefficients and filter parameters of the post-filter are selected such that the post-filter is an exact inverse of the pre-filter. ### Claim 4 (depends on 2) The method of claim 2, wherein at least one of the pre-filter and the post-filter comprise different filters, and chroma components of the picture are filtered using different ones of the different filters than the luma components of the picture. ### Claim 5 (depends on 2) The method of claim 2, wherein a post-filter transform matrix used by the post-filter to filter the reconstructed data is decomposed into two summands, one of the two summands being an identity matrix and another one of the two summands being a matrix representing a multiplication of a first, a second, and a third matrix, the first and the third matrices being fixed, and the second matrix being variable in order to reverse an adaptation of the pre-filter. ### Claim 6 (depends on 2) The method of claim 2, wherein filter coefficients and filter parameters of the post-filter are selected such that the post-filter is an inverse of the pre-filter. ### Claim 7 (depends on 6) The method of claim 6, wherein the post-filter is configured to provide a same output data there from as input data provided to the pre-filter by minimizing a difference between an observation and a pre-filtered estimate, the observation relating to input data provided to the post-filter, and the pre-filtered estimate relating to an estimate of the input data for the picture prior to filtering by the pre-filter. ### Claim 8 (depends on 2) The method of claim 2, wherein the pre-filter and the post-filter are integer implementations determined so as to minimize a distance relating to an exact invertibility between the pre-filter and the post-filter. ### Claim 9 (depends on 2) The method of claim 2, wherein a filter size of the pre-filter and the post-filter is a same size as a transform applied to residue data, the residue data representing a difference between the input data for the picture and reference data for at least one reference picture. ### Claim 10 (depends on 9) The method of claim 9, wherein the pre-filter and the post-filer comprise multiple filters, and at least one of the multiple filters is applied to all transform sizes of transforms applied to the residue data. ### Claim 11 (depends on 2) The method of claim 2, wherein a filter size of at least one of the pre-filter and the post-filter is different from a size of a transform applied to the input data. ### Claim 12 (depends on 2) The method of claim 2, wherein the pre-filter and the post-filter are applied to only a portion of the input data. ### Claim 13 (depends on 12) The method of claim 12, wherein the portion of the input data is selected from at least one of a block boundary, and within a block. ### Claim 14 An apparatus, comprising: a video decoder for decoding residual image data for a picture, wherein said video decoder comprises a pre-filter and a post-filter coupled to the pre-filter, wherein said pre-filter filters a reference picture for use in decoding the residual image data and the post-filter filters in-loop reconstructed data for the picture, wherein at least one of filter coefficients and filter parameters for at least one of the pre-filter and the post-filter are selected responsive to at least one of a resolution, a local gradient, and a gradient direction, wherein gradient is a difference between pixels being pre-filtered, and adaptation is made discrete by partitioning a gradient strength, gradient direction and gradient variance into disjoint ranges and applying fixed filters for different ranges, and wherein a Lagrangian parameter for luminance and chrominance in the video decoder's rate-distortion optimizer is a function of the coding efficiency of the pre-filter and the post-filter; and wherein the pre-filter and post-filter are implemented with a lifting scheme and adaptation is embedded therein by changing values of a diagonal matrix, obtained by performing singular value decomposition on a fixed pre-filter transform matrix, such that values of the diagonal matrix can be reversed from those of an encoder. ### Claim 15 In a video decoder, a method comprising: decoding residual image data for a picture, wherein said video decoder comprises a pre-filter and a post-filter coupled to the pre-filter, wherein said pre-filter filters a reference picture for use in decoding the residual image data and the post-filter filters in-loop reconstructed data for the picture, wherein at least one of filter coefficients and filter parameters for at least one of the pre-filter and the post-filter are selected responsive to at least one of a resolution, a local gradient, and a gradient direction, wherein gradient is a difference between pixels being pre-filtered, and adaptation is made discrete by partitioning a gradient strength, gradient direction and gradient variance into disjoint ranges and applying fixed filters for different ranges, and wherein a Lagrangian parameter for luminance and chrominance in the video decoder's rate-distortion optimizer is a function of the coding efficiency of the pre-filter and the post-filter; and wherein the pre-filter and post-filter are implemented with a lifting scheme and adaptation is embedded therein by changing values of a diagonal matrix, obtained by performing singular value decomposition on a fixed pre-filter transform matrix, such that values of the diagonal matrix can be reversed from those of an encoder. ### Claim 16 (depends on 15) The method of claim 15, wherein filter coefficients and filter parameters of the post-filter are selected such that the post-filter is an exact inverse of the pre-filter. ### Claim 17 (depends on 15) The method of claim 15, wherein at least one of the pre-filter and the post-filter comprise different filters, and chroma components of the picture are filtered using different ones of the different filters than the luma components of the picture. ### Claim 18 (depends on 15) The method of claim 15, wherein at least one of filter coefficients and filter parameters for at least one of the pre-filter and the post-filter are selected responsive to at least one of a resolution, a quantization level, a local gradient, a prediction mode, and a gradient direction. ### Claim 19 (depends on 15) The method of claim 15, wherein a post-filter transform matrix used by the post-filter to filter the reconstructed data is decomposed into two summands, one of the two summands being an identity matrix and another one of the two summands being a matrix representing a multiplication of a first, a second, and a third matrix, the first and the third matrices being fixed, and the second matrix being variable in order to reverse an adaptation of the pre-filter. ### Claim 20 (depends on 15) The method of claim 15, wherein filter coefficients and filter parameters of the post-filter are selected such that the post-filter is an inverse of the pre-filter. ### Claim 21 (depends on 20) The method of claim 20, wherein the post-filter is configured to provide a same output data there from as input data provided to the pre-filter by minimizing a difference between an observation and a pre-filtered estimate, the observation relating to input data provided to the post-filter, and the pre-filtered estimate relating to an estimate of the input data for the picture prior to filtering by the pre-filter. ### Claim 22 (depends on 15) The method of claim 15, wherein the pre-filter and the post-filter are integer implementations determined so as to minimize a distance relating to an exact invertibility between the pre-filter and the post-filter. ### Claim 23 (depends on 15) The method of claim 15, wherein a filter size of the pre-filter and the post-filter is a same size as a transform applied to residue data, the residue data representing a difference between the input data for the picture and reference data for at least one reference picture. ### Claim 24 (depends on 23) The method of claim 23, wherein the pre-filter and the post-filer comprise multiple filters, and at least one of the multiple filters is applied to all transform sizes of transforms applied to the residue data. ### Claim 25 (depends on 15) The method of claim 15, wherein a filter size of at least one of the pre-filter and the post-filter is different from a size of a transform applied to the input data. ### Claim 26 (depends on 15) The method of claim 15, wherein the pre-filter and the post-filter are applied to only a portion of the input data. ### Claim 27 (depends on 26) The method of claim 26, wherein the portion of the input data is selected from at least one of a block boundary, and within a block. ### Claim 28 A non-transitory computer readable storage media having video signal data encoded thereupon, comprising: input data for a picture encoded, wherein the input data for the picture, when encoded, was pre-filtered using a pre-filter, and in-loop reconstructed data for the picture, when encoded, was post-filtered using a post-filter directly coupled to the pre-filter, wherein at least one of filter coefficients and filter parameters for at least one of the pre-filter and the post-filter are selected responsive to at least one of a resolution, a local gradient, and a gradient direction, wherein gradient is a difference between pixels being pre-filtered, and adaptation is made discrete by partitioning a gradient strength, gradient direction and gradient variance into disjoint ranges and applying fixed filters for different ranges, and wherein a Lagrangian parameter for luminance and chrominance in the encoded video's rate-distortion optimizer is a function of the coding efficiency of the pre-filter and the post-filter; and wherein the pre-filter and post-filter are implemented with a lifting scheme and adaptation is embedded therein by changing values of a diagonal matrix, obtained by performing singular value decomposition on a fixed pre-filter transform matrix, such that values of the diagonal matrix can be reversed from those of an encoder.