Handling video unit boundaries and virtual boundaries

An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

• determine, for a conversion between a picture of a video that comprises one or more coding tree blocks and a bitstream of the video, whether a line buffer virtual boundary handling is enabled for a coding tree block within the picture for a first filtering process based on a relationship between a bottom boundary of the coding tree block and a bottom boundary of the picture;
• and
• perform the conversion based on determining whether the line buffer virtual boundary handling is enabled,
• wherein the first filtering process comprises:
• deciding at least one filtering index for the coding tree block;
• deriving a filtering coefficient set based on the at least one filtering index;
• and
• performing a filtering operation based on the filtering coefficient set,
• wherein the line buffer virtual boundary handling is enabled, in a case that a) the bottom boundary of the coding tree block is not the bottom boundary of the picture containing the coding tree block and b) the bottom boundary of the coding tree block is a bottom boundary of a brick and the first filtering process across boundaries of bricks is disabled,
• wherein the line buffer virtual boundary handling is enabled, in a case that a) the bottom boundary of the coding tree block is not the bottom boundary of the picture containing the coding tree block and b) the bottom boundary of the coding tree block is a bottom boundary of a slice and the first filtering process across boundaries of slices is disabled.

A non-transitory computer-readable storage medium storing instructions that cause a processor to:

• determine, for a conversion between a picture of a video that comprises one or more coding tree blocks and a bitstream of the video, whether a line buffer virtual boundary handling is enabled for a coding tree block within the picture for a first filtering process based on a relationship between a bottom boundary of the coding tree block and a bottom boundary of the picture;
• and
• perform the conversion based on determining about whether the line buffer virtual boundary handling is enabled,
• wherein the first filtering process comprises:
• deciding at least one filtering index for the coding tree block;
• deriving a filtering coefficient set based on the at least one filtering index;
• and
• performing a filtering operation based on the filtering coefficient set,
• wherein the line buffer virtual boundary handling is enabled in a case that a) the bottom boundary of the coding tree block is not the bottom boundary of the picture containing the coding tree block and b) the bottom boundary of the coding tree block is a bottom boundary of a brick and the first filtering process across boundaries of bricks is disabled,
• wherein the line buffer virtual boundary handling is enabled, in a case that a) the bottom boundary of the coding tree block is not the bottom boundary of the picture containing the coding tree block and b) the bottom boundary of the coding tree block is a bottom boundary of a slice and the first filtering process across boundaries of slices is disabled.

A method for storing a bitstream of a video, comprising:

• determining whether a line buffer virtual boundary handling is enabled for a coding tree block within a picture of a video for a first filtering process based on a relationship between a bottom boundary of the coding tree block and a bottom boundary of the picture;
• generating the bitstream based on the determining, and
• storing the bitstream in a non-transitory computer-readable recording medium,
• wherein the first filtering process comprises:
• deciding at least one filtering index for the coding tree block;
• deriving a filtering coefficient set based on the at least one filtering index;
• and
• performing a filtering operation based on the filtering coefficient set,
• wherein the line buffer virtual boundary handling is enabled in a case that a) the bottom boundary of the coding tree block is not the bottom boundary of the picture containing the coding tree block and b) the bottom boundary of the coding tree block is a bottom boundary of a brick and the first filtering process across boundaries of bricks is disabled,
• wherein the line buffer virtual boundary handling is enabled, in a case that a) the bottom boundary of the coding tree block is not the bottom boundary of the picture containing the coding tree block and b) the bottom boundary of the coding tree block is a bottom boundary of a slice and the first filtering process across boundaries of slices is disabled.