Method and apparatus for in-loop filtering

#The present invention relates to video coding system. In particular, the present invention relates to method and apparatus for implementation of in-loop filtering associated with a video encoder or decoder.

#Motion estimation is an effective inter-frame coding technique to exploit temporal redundancy in video sequences. Motion-compensated inter-frame coding has been widely used in various international video coding standards. The motion estimation adopted in various coding standards is often a block-based technique, where motion information such as coding mode and motion vector is determined for each macroblock or similar block configuration. In addition, intra-coding is also adaptively applied, where the picture is processed without reference to any other picture. The inter-predicted or intra-predicted residues are usually further processed by transformation, quantization, and entropy coding to generate a compressed video bitstream. During the encoding process, coding artifacts are introduced, particularly in the quantization process. In order to alleviate the coding artifacts, additional processing has been applied to reconstructed video to enhance picture quality in newer coding systems. The additional processing is often configured in an in-loop operation so that the encoder and decoder may derive the same reference pictures to achieve improved system performance.

#In the High Efficiency Video Coding (HEVC) standard being developed, Deblocking Filter (DF), Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF) have been developed to enhance picture quality. The in-loop filter information may have to be incorporated in the bitstream so that a decoder can properly recover the required information. Therefore, in-loop filter information from SAO and ALF is usually provided to entropy encoder for incorporation into the bitstream. In HEVC, DF is applied to the reconstructed video, where the horizontal deblocking filter is first applied to the reconstructed video data across vertical block boundaries and the vertical deblocking filter is then applied to the horizontally DF-processed (also referred as H-DF-processed or horizontal-deblocked) video data across horizontal block boundaries. After both horizontal DF and vertical DF filtering are applied, the fully DF-processed video data is processed by SAO. ALF is then applied to the SAO-processed video data. While the loop processing order for HEVC is from DF to SAO, and then to ALF, the processing order maybe different among various loop filters in other video systems.

#FIG. 1 illustrates an exemplary adaptive video coding system incorporating DF, SAO and ALF in-loop processing.

#FIG. 2A illustrates an example of a vertical block boundary with 4 boundary pixels on each side of the block boundary.

#FIG. 2B illustrates an example of a horizontal block boundary with 4 boundary pixels on each side of the block boundary.

#FIG. 9 illustrates an example of processing flow according to an embodiment of the present invention, where the luma line buffers required for DF, SAO, and ALF with LCU-based decoding are shown. A given current LCU is located at the upper right quadrant of the horizontal LCU boundary 910 and vertical LCU boundary 912. Lines A through J are first processed by horizontal DF and then by vertical DF. As mentioned before, the horizontal DF for lines K through N is delayed until the lower LCU arrives in order to avoid line buffers of horizontal-deblocked pixels for the vertical DF for lines K through N. Therefore, four lines (lines K through N) of pre-DF pixels are stored for DF processing when the lower LCU becomes available. Next, SAO is ready to apply to DF output pixels. Since the vertical DF for lines K through N will not change line K, horizontal DF can be additionally applied to line K (i.e., boundary line pn−1). Accordingly, SAO can be additionally applied to line J, as illustrated by the 3×3 window 930 due to the availability of the additional DF-processed line K. When horizontal DF is applied to line K, the horizontal-deblocked output pixels of line K will not be stored in the line buffers in order to save line buffers. When the horizontal-deblocked pixels of line K is needed for vertical DF process, the horizontal-deblocked pixels of line K can be generated again. This is not an issue for hardware based implementation. After SAO processes lines A through J, the 4×4 block properties, as illustrated by the 4×4 window 950, can be calculated based on the 5×5 window 952. At this moment, ALF can process lines A through H. After this, no further process can be done for the current LCU until the lower LCU becomes available.

#When the lower LCU arrives, lines K through P are first processed by DF and then by SAO. The horizontal DF is applied to the bottom three lines (i.e., lines L through N) of the current LCU (after the lower LCU arrives, the LCU located at the upper-right quadrant of LCU boundaries 910 and 912 is still referred as the “current LCU”) and then the vertical DF (as indicated by box 920 as one example) can be applied to the horizontal-deblocked pixels. When SAO processes line K, line J is required as indicated by the 3×3 window 930. Therefore, one line (i.e., line J) of DF output pixels has to be stored for SAO. Next, the 4×4 block properties for lines I through P are calculated. Finally, lines I through P can be filtered by ALF. When line I is filtered by ALF, it requires lines G through K, as illustrated by the filter footprint 940 for underlying pixel 942. However, calculating properties for the 4×4 block (indicated by window 950) associated with lines I and J needs SAO processed data for lines F through J as indicated by window 952. Therefore, five lines (i.e., line F through J) of SAO output pixels have to be stored for ALF. Accordingly, the total in-loop filtering requires ten luma line buffers, which is one line buffer less than the method associated with FIG. 7. Since the Intra luma prediction already stores one line (i.e., line N) of pre-DF pixels, this luma line buffer corresponds to boundary pixel line p0 and can be shared with in-loop filtering. When the coded data for the LCU below arrives, the residues for the LCU below can be recovered using the Entropy decoder 142 as shown in FIG. 1. For Intra coded video data, the reconstructed video data for the LCU below can be recovered based on reconstructed line p0 and the recovered residues for the LCU below. While it is not explicitly described, it is understood that Intra prediction decoding for the LCU below may also require reconstructed data immediately adjacent to the left LCU boundary of the LCU below. Furthermore, if a filter index line buffer is used to store BA mode filter selections for lines G through J, there is no need for computing the block properties again during filtering of lines I through J. In this way, one line (i.e., line F) of SAO output pixels can be saved for ALF. The filter index line buffer requires only 4 bits per 4×4 block, which is only about 10% in comparison with the pixel line buffer.

#In the above example, one line (i.e., line J) of DF output pixels has to be stored in line buffers for SAO. The DF output pixels are stored for EO classification. An example of EO classification based on HEVC HM4.0 is shown in Table 1, where the center pixel value is compared with two neighboring pixel values. In other words, the EO classification can be determined according to the sign of the difference between the center pixel value and a respective neighboring pixel value. Therefore, partial derived information instead of actual pixel value can be stored. In the above example, the sign information (partial derived information) between line J and line I can be stored instead of the DF-processed pixel values for line J. Depending on the specific in-loop filters used, partial derived information can be determined Furthermore, depending on the underlying in-loop filter, the partial derived information may be derived from reconstructed video data, horizontal-deblocked video data, fully DF processed data, or SAO processed data.