Classification in adaptive loop filtering

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

• determine, for a conversion between a picture in a video comprising one or more coding tree blocks and a bitstream of the video, gradients of a subset of samples in a region with a dimension of 8×8 for deriving a classification of a 4×4 block of a first coding tree block in a first filtering process, wherein the 4×4 block is located within the first coding tree block and the 4×4 block is located within the 8×8 region;
• and
• perform the conversion based on the determination,
• wherein the first filtering process comprises:
• determining a filtering coefficient set for a sample of the 4×4 block based on the gradients;
• determining a clipping value set for the sample;
• and
• applying a filter on the sample based on the filtering coefficient set and the clipping value set,
• wherein in case it is disallowed to use samples across a boundary of a video unit in the first filtering process, left-most two columns of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a left boundary of the video unit;
• or top-most two lines of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a top boundary of the video unit;
• or right-most two lines of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a right boundary of the video unit;
• or lowest two lines of padded samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a bottom boundary of the video unit.

A non-transitory computer-readable storage medium storing instructions that cause a processor to:

• determine, for a conversion between a picture in a video comprising one or more coding tree blocks and a bitstream of the video, gradients of a subset of samples in a region with a dimension of 8×8 for deriving a classification of a 4×4 block of a first coding tree block in a first filtering process, wherein the 4×4 block is located within the first coding tree block and the 4×4 block is located within the 8×8 region;
• and
• perform the conversion based on the determination,
• wherein the first filtering process comprises:
• determining a filtering coefficient set for a sample of the 4×4 block based on the gradients;
• determining a clipping value set for the sample;
• and
• applying a filter on the sample based on the filtering coefficient set and the clipping value set,
• wherein in case it is disallowed to use samples across a boundary of a video unit in the first filtering process, left-most two columns of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a left boundary of the video unit;
• or top-most two lines of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a top boundary of the video unit;
• or right-most two lines of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a right boundary of the video unit;
• or lowest two lines of padded samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a bottom boundary of the video unit.

A method for storing a bitstream of a video, comprising:

• determining gradients of a subset of samples in a region with a dimension of 8×8 for deriving a classification of a 4×4 block of a first coding tree block in a picture of the video in a first filtering process, wherein the 4×4 block is located within the first coding tree block and the 4×4 block is located within the 8×8 region;
• generating the bitstream based on the determining;
• and
• storing the bitstream in a non-transitory computer-readable recording medium,
• wherein the first filtering process comprises:
• determining a filtering coefficient set for a sample of the 4×4 block based on the gradients;
• determining a clipping value set for the sample;
• and
• applying a filter on the sample based on the filtering coefficient set and the clipping value set,
• wherein in case it is disallowed to use samples across a boundary of a video unit in the first filtering process, left-most two columns of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a left boundary of the video unit;
• or top-most two lines of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a top boundary of the video unit;
• or right-most two lines of samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a right boundary of the video unit;
• or lowest two lines of padded samples in the 8×8 region are padded when there is a sample in the 4×4 block located at a bottom boundary of the video unit.