# US 2021/0344930 A1 · Macroblocking artifact detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0344930 A1
- Document ID: US20210344930A1
- Status: Published application
- Kind: A1
- Application number: 17/237,722
- Filed: 2021-04-22
- Published: 2021-11-04
- Earliest claimed: 2020-05-04
- Assignees: Ssimwave Inc.
- Applicants: Ssimwave Inc.
- Inventors: Zhou WANG; Kai ZENG
- Web page: https://patentgrep.com/patent/US20210344930A1

## CPC

- H04N 19/14: Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence)
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/172: the region being a picture, frame or field
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/86: involving reduction of coding artifacts, e.g. of blockiness
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Macroblocking artifacts are assessed in an encoded video. Edges are detected and edge strengths are computed at macroblock boundaries in each decoded frame of the video. The detected edges are paired at horizontal or vertical sides of each macroblock to identify macroblock edge pairs. A macroblocking level of the frame is identified according to the macroblock edge pairs. For instance, for each of the macroblock edge pairs, the edges of the pair are identified as being macroblocking edges if edge strengths of both sides of the pair exceed a significance threshold, and the macroblocking level of the frame is determined by aggregating the macroblocking edges in the frame.

## Claims

### Claim 1

A method for assessing macroblocking artifacts in an encoded video, comprising:
detecting edges and computing edge strengths at macroblock boundaries in each decoded frame of the video;
pairing the detected edges at horizontal or vertical sides of each macroblock to identify macroblock edge pairs; and
identifying a macroblocking level of the frame according to the macroblock edge pairs.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
for each of the macroblock edge pairs, identifying the edges of the pair as being macroblocking edges if edge strengths of both sides of the pair exceed a significance threshold; and
determining the macroblocking level of the frame by aggregating the macroblocking edges in the frame.

### Claim 3 (depends on 1)

The method of claim 1, further comprising determining macroblock sizes and boundary positions according to aspects of an encoding configuration used to generate the encoded video.

### Claim 4 (depends on 1)

The method of claim 1, further comprising computing the edge strength according to a jump in pixel intensity at the edge.

### Claim 5 (depends on 4)

The method of claim 4, further comprising scaling the edge strength by a scale factor indicative of human visual luminance masking effect, where the scale factor is determined by the pixel intensity or luminance level at the edge.

### Claim 6 (depends on 4)

The method of claim 4, further comprising scaling the edge strength by a scale factor indicative of human visual texture masking effect, where the scale factor is determined by a local signal energy or activity measure in proximity to the edge.

### Claim 7 (depends on 1)

The method of claim 1, further comprising determining whether the edge strength is significant by comparing it with a pre-defined visibility threshold.

### Claim 8 (depends on 7)

The method of claim 7, further comprising defining the visibility threshold as a human visual just noticeable difference (JND) or as the JND multiplied by a scale factor.

### Claim 9 (depends on 1)

The method of claim 1, further comprising creating a macroblocking map for each video frame by:
marking pixel positions of detected macroblocking edges; and
determining an intensity level of each position in the macroblocking map as a measure of the edge strength at that position.

### Claim 10 (depends on 9)

The method of claim 9, further comprising aggregating the macroblocking map to a frame-level macroblocking strength measure, by computing one or more of average, weighted-average, median, percentile, maximum, or minimum of the intensity level of each position in the macroblocking map.

### Claim 11 (depends on 9)

The method of claim 9, further comprising aggregating the macroblocking map to a frame-level macroblocking spread measure, by computing a percentage of area in the frame that contains macroblocking edges compared to overall area of the frame.

### Claim 12 (depends on 9)

The method of claim 9, further comprising creating a frame-level overall macroblocking level measure by:
aggregating the macroblocking map to a frame-level macroblocking strength measure, by computing one or more of average, weighted-average, median, percentile, maximum, or minimum of the intensity level of each position in the macroblocking map;
aggregating the macroblocking map to a frame-level macroblocking spread measure, by computing a percentage of area in the frame that contains macroblocking edges compared to overall area of the frame; and
combining the frame-level macroblocking strength measure and the frame-level macroblocking spread measure using computations including one or more of sum, average, product, weighted-average, median, maximum, or minimum.

### Claim 13 (depends on 1)

The method of claim 1, further comprising aggregating per-frame macroblocking levels to macroblocking levels at over a multi-frame time scale, the time scale including one or more of per group-of-picture, per-scene, per video asset, per-second, per-minute, per-hour, or per arbitrary time period.

### Claim 14

A system for assessing macroblocking artifacts in an encoded video, comprising:
a computing device programmed to:
detect edges and compute edge strengths at macroblock boundaries in each decoded frame of the video;
pair the detected edges at horizontal or vertical sides of each macroblock to identify macroblock edge pairs; and
identify a macroblocking level of the frame according to the macroblock edge pairs.

### Claim 15 (depends on 14)

The system of claim 14, wherein the computing device is further programmed to:
for each of the macroblock edge pairs, identify the edges of the pair as being macroblocking edges if edge strengths of both sides of the pair exceed a significance threshold; and
determine a macroblocking level of the frame by aggregating the macroblocking edges in the frame.

### Claim 16 (depends on 14)

The system of claim 14, wherein the computing device is further programmed to determine macroblock sizes and boundary positions according to aspects of an encoding configuration used to generate the encoded video.

### Claim 17 (depends on 14)

The system of claim 14, wherein the computing device is further programmed to compute the edge strength according to a jump in pixel intensity at the edge.

### Claim 18 (depends on 17)

The system of claim 17, wherein the computing device is further programmed to scale the edge strength by a scale factor indicative of human visual luminance masking effect, where the scale factor is determined by the pixel intensity or luminance level at the edge.

### Claim 19 (depends on 17)

The system of claim 17, wherein the computing device is further programmed to scale the edge strength by a scale factor indicative of human visual texture masking effect, where the scale factor is determined by a local signal energy or activity measure in proximity to the edge.

### Claim 20 (depends on 14)

The system of claim 14, wherein the computing device is further programmed to determine whether the edge strength is significant by comparing it with a pre-defined visibility threshold.

### Claim 21 (depends on 20)

The system of claim 20, wherein the computing device is further programmed to define the visibility threshold as a human visual just noticeable difference (JND) or as the JND multiplied by a scale factor.

### Claim 22 (depends on 14)

The system of claim 14, wherein the computing device is further programmed to create a macroblocking map for each video frame by operations including to:
mark pixel positions of detected macroblocking edges; and
determine an intensity level of each position in the macroblocking map as a measure of edge strength at that position.

### Claim 23 (depends on 22)

The system of claim 22, wherein the computing device is further programmed to aggregate the macroblocking map to a frame-level macroblocking strength measure, by computing one or more of average, weighted-average, median, percentile, maximum, or minimum of the intensity level of each position in the macroblocking map.

### Claim 24 (depends on 22)

The system of claim 22, wherein the computing device is further programmed to aggregate the macroblocking map to a frame-level macroblocking spread measure, by computing a percentage of area in the frame that contains macroblocking edges compared to overall area of the frame.

### Claim 25 (depends on 22)

The system of claim 22, wherein the computing device is further programmed to create a frame-level overall macroblocking level measure by operations including to:
aggregate the macroblocking map to a frame-level macroblocking strength measure, by computing one or more of average, weighted-average, median, percentile, maximum, or minimum of the intensity level of each position in the macroblocking map;
aggregate the macroblocking map to a frame-level macroblocking spread measure, by computing a percentage of area in the frame that contains macroblocking edges compared to overall area of the frame; and
combine the frame-level macroblocking strength measure and the frame-level macroblocking spread measure using computations including one or more of sum, average, product, weighted-average, median, maximum, or minimum.

### Claim 26 (depends on 14)

The system of claim 14, wherein the computing device is further programmed to aggregate per-frame macroblocking levels to macroblocking levels at over a multi-frame time scale, the time scale including one or more of per group-of-picture, per-scene, per video asset, per-second, per-minute, per-hour, or per arbitrary time period.