# US 2012/0250772 A1 · Multi-threaded implementations of deblock filtering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0250772 A1
- Document ID: US20120250772A1
- Status: Published application
- Kind: A1
- Application number: 13/078,852
- Filed: 2011-04-01
- Published: 2012-10-04
- Earliest claimed: 2011-04-01
- Assignees: Microsoft Corporation
- Inventors: Wenfeng Gao; Yongjun Wu; Shyam Sadhwani
- Web page: https://patentgrep.com/patent/US20120250772A1

## CPC

- H04N 19/865: {with detection of the former encoding block subdivision in decompressed video}
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/127: Prioritisation of hardware or computational resources
- H04N 19/157: Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N 19/159: Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
- H04N 19/16: for a given display mode, e.g. for interlaced or progressive display mode
- H04N 19/172: the region being a picture, frame or field
- H04N 19/436: using parallelised computational arrangements
- H04N 19/82: involving filtering within a prediction loop
- H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Multi-threaded implementations of deblock filtering improve encoding and/or decoding efficiency. For example, a video encoder or decoder partitions a video picture into multiple segments. The encoder/decoder selects between multiple different patterns for splitting operations of deblock filtering into multiple passes. The encoder/decoder organizes the deblock filtering as multiple tasks, where a given task includes the operations of one of the passes for one of the segments. The encoder/decoder then performs the tasks with multiple threads. The performance of the tasks is constrained by task dependencies which, in general, are based at least in part on which lines of the picture are in the respective segments and which deblock filtering operations are in the respective passes. The task dependencies can include a cross-pass, cross-segment dependency between a given pass of a given segment and an adjacent pass of an adjacent segment.

## Claims

### Claim 1

In a computing system that implements a video encoder and/or a video decoder, a method of performing deblock filtering, the method comprising:
partitioning a video picture into multiple segments for deblock filtering whose operations are split into multiple passes;
organizing the deblock filtering for the video picture as multiple tasks, wherein a given task of the multiple tasks includes the operations of one of the multiple passes for one of the multiple segments; and
performing the multiple tasks with multiple threads, wherein the performance of the multiple tasks is constrained by task dependencies that include a cross-pass, cross-segment dependency.

### Claim 2 (depends on 1)

The method of claim 1 wherein the deblock filtering is in-loop deblock filtering or post-processing deblock filtering, wherein segment boundaries for the multiple segments align with block boundaries of blocks of the picture, wherein each of the multiple segments includes one or more rows or columns of the blocks of the video picture.

### Claim 3 (depends on 1)

The method of claim 1 further comprising:
determining picture coding type and frame coding mode of the video picture; and
selecting between multiple different patterns for splitting the operations of the deblock filtering based at least in part on the picture coding type and based at least in part on the frame coding mode.

### Claim 4 (depends on 1)

The method of claim 1 further comprising setting a count for the multiple segments based at least in part on a count of the multiple threads that are to be used for the deblock filtering.

### Claim 5 (depends on 1)

The method of claim 1 wherein the deblock filtering is in-loop deblock filtering, and wherein the multiple passes comprise:
a first pass that includes making at least some filtering decisions per segment;
a second pass that includes filtering of any horizontal block boundaries to be filtered;
a third pass that includes filtering of any horizontal sub-block boundaries to be filtered; and
a fourth pass that includes filtering of any vertical boundaries to be filtered.

### Claim 6 (depends on 5)

The method of claim 5 wherein, according to the task dependencies:
for the first pass, the multiple segments are independent of each other;
for the second pass, a given segment is dependent on completion of the first pass for the given segment and dependent on completion of the first pass for a first adjacent segment;
for the third pass, the given segment is dependent on completion of the second pass for the given segment and dependent on completion of the second pass for a second adjacent segment; and
for the fourth pass, the multiple segments are independent of each other, but the given segment is dependent on completion of the third pass for the given segment.

### Claim 7 (depends on 1)

The method of claim 1 wherein the deblock filtering is in-loop deblock filtering, wherein the video picture is an interlaced video frame, and wherein the multiple passes comprise:
a first pass that includes making at least some filtering decisions per segment;
a second pass that includes filtering of any horizontal boundaries to be filtered for a top field of the interlaced video frame;
a third pass that includes filtering of any horizontal boundaries to be filtered for a bottom field of the interlaced video frame; and
a fourth pass that includes filtering of any vertical boundaries to be filtered for the top field and the bottom field.

### Claim 8 (depends on 7)

The method of claim 7 wherein, according to the task dependencies:
for the first pass, the multiple segments are independent of each other;
for the second pass, a given segment is dependent on completion of the first pass for the given segment, dependent on completion of the second pass for a first adjacent segment, and dependent on completion of the first pass for a second adjacent segment;
for the third pass, the given segment is dependent on completion of the first pass for the given segment, dependent on completion of the third pass for the first adjacent segment, and dependent on completion of the first pass for the second adjacent segment, the multiple segments for the second pass being independent of the multiple segments for the third pass; and
for the fourth pass, the multiple segments are independent of each other, but the given segment is dependent on completion of the second pass and the third pass for the given segment.

### Claim 9 (depends on 1)

The method of claim 1 further comprising determining the task dependencies based at least in part on which lines of the video picture are in the respective multiple segments and which operations of the deblock filtering are in the respective multiple passes.

### Claim 10 (depends on 1)

The method of claim 1 wherein the cross-pass, cross-segment dependency is between a given pass of a given segment and an adjacent pass of an adjacent segment.

### Claim 11 (depends on 10)

The method of claim 10 wherein the cross-pass, cross-segment dependency is due to use, during the given pass of the given segment, of a loop filter flag set in the adjacent pass of the adjacent segment.

### Claim 12 (depends on 10)

The method of claim 10 wherein the cross-pass, cross-segment dependency is due to use, during the given pass of the given segment, of pixel values set in the adjacent pass of the adjacent segment.

### Claim 13

A computing system that implements a video encoder and/or a video decoder, wherein the computing system includes a processing unit, memory and storage media storing computer-executable instructions for causing the computing system to perform a method comprising:
determining picture coding type of a video picture;
partitioning the video picture into multiple segments for in-loop deblock filtering;
based at least in part on the picture coding type, selecting between multiple different patterns for splitting operations of the in-loop deblock filtering into multiple passes;
organizing the in-loop deblock filtering for the video picture as multiple tasks, a given task of the multiple tasks including the operations of one of the multiple passes for one of the multiple segments; and
performing the multiple tasks with multiple threads.

### Claim 14 (depends on 13)

The computing system of claim 13 wherein the method further comprises:
determining frame coding mode of the video picture, wherein the selecting is also based at least in part on the frame coding mode of the video picture.

### Claim 15 (depends on 14)

The computing system of claim 14 wherein the multiple different patterns include:
(a) a first pattern selected if the frame coding mode is progressive and the picture coding type is P, or the frame coding mode is interlaced field and the picture coding type is P or B;
(b) a second pattern selected if the frame coding mode is progressive and the picture coding type is I, B or BI, or the frame coding mode is interlaced field and the picture coding type is I or BI; and
(c) a third pattern selected if the frame coding mode is interlaced frame.

### Claim 16 (depends on 13)

The computing system of claim 13 wherein, for a given segment of the multiple segments, the multiple passes comprise:
a first pass that includes filtering of horizontal boundaries in the given segment; and
a second pass that includes filtering of vertical boundaries in the given segment.

### Claim 17

A computing system that implements a video encoder and/or a video decoder, wherein the computing system includes a processing unit and memory, the computing system being adapted to perform a method comprising:
partitioning a video picture into multiple segments for deblock filtering whose operations are split into three or more passes;
organizing the deblock filtering for the video picture as multiple tasks, a given task of the multiple tasks including the operations of one of the passes for one of the multiple segments; and
performing the multiple tasks with multiple threads.

### Claim 18 (depends on 17)

The computing system of claim 17 wherein the three or more passes comprise a first pass for making filtering decisions for the multiple segments.

### Claim 19 (depends on 18)

The computing system of claim 18 wherein the three or more passes further comprise:
a second pass for filtering of any horizontal block boundaries to be filtered;
a third pass for filtering of any horizontal sub-block boundaries to be filtered; and
a fourth pass for filtering of any vertical boundaries to be filtered.

### Claim 20 (depends on 18)

The computing system of claim 18 wherein the three or more passes further comprise:
a second pass for filtering of any horizontal boundaries to be filtered for a top field;
a third pass for filtering of any horizontal boundaries to be filtered for a bottom field; and
a fourth pass for filtering of any vertical boundaries to be filtered for the top field and the bottom field.
