# US 12,002,186 B1 · Surround area detection and blending for image filtering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,186 B1
- Document ID: US12002186B1
- Status: Granted
- Kind: B1
- Application number: 18/568,104
- Filed: 2022-06-01
- Published: 2024-07-04
- Granted: 2024-06-04
- Earliest claimed: 2021-06-11
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Tsung-Wei Huang; Guan-Ming Su
- Web page: https://patentgrep.com/patent/US12002186B1

## CPC

- G06T 5/20: using local operators
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06T 5/40: using histogram techniques
- G06T 5/70: Denoising; Smoothing
- G06V 10/764: using classification, e.g. of video objects
- G06T 2207/20192: Edge enhancement; Edge preservation
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Methods and systems to detect and blend a surround area to avoid filtering artifact due to the area in image filtering are disclosed. The described methods include a compensation step and can be applied to arbitrary images with padded areas of arbitrary shape, such as letterboxes, pillar-boxes, ovals, or other shapes, including logos and close captions. Such methods detect the surround areas in the image with possible compression artifact and noise, and then perform blending to minimize the effects of the surround areas for any arbitrary image filtering operations.

## Claims

### Claim 1

A method for detecting a non-active area in an image, the method comprising:
calculating a histogram of a predefined area located at a boundary of the image;
finding a peak and a width of the peak of the histogram;
based on the peak and the width of the peak of the histogram, classifying a presence of the non-active area in the image, thereby generating a peak detection score;
based on a ratio of the pixels that belong to the peak on a predefined minimum possible non-active areas at the predefined area located at the boundary of the image, classifying the presence of the non-active area in the image, thereby generating a boundary detection score;
generating a total score based on a combination of the peak and the boundary detection scores, and
detecting the presence of the non-active area based on the total score.

### Claim 2 (depends on 1)

The method of claim 1, further comprising modeling the peak of the histogram using a smoothing filter to represent the non-active area as a non-active area mask.

### Claim 3 (depends on 2)

The method of claim 2, wherein the smoothing filter comprises Gaussian kernels.

### Claim 4 (depends on 2)

The method of claim 2, further comprising generating a non-active area probability map based on the non-active area mask, wherein the non-active area probability map provides, for all pixels in the image, a probability that a pixel of the image belongs to the non-active area.

### Claim 5 (depends on 4)

The method of claim 4, wherein the generating the non-active area probability map includes:
calculating a probability using a scaled piecewise Gaussian kernel, and
back-projecting the probability to generate the non-active area probability map.

### Claim 6 (depends on 4)

The method of claim 4 further comprising blending the non-active area and a main subject of the image based on the non-active area probability map.

### Claim 7 (depends on 6)

The method of claim 6, wherein the blending is performed using a weighted gaussian filtering.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
calculating a blending weight based on the non-active area probability map, and
generating a weighted image based on the image and the blending weight.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
gaussian filtering the blending weight to generate a filtered weight;
gaussian filtering the weighted image to generate a filtered weighted image, and
generating a weighted-filtered image based on the filtered weighted image and the filtered weight.

### Claim 10 (depends on 9)

The method of claim 9, further comprising generating a blended image using a weighted summation of the image and the weighted-filtered image.

### Claim 11 (depends on 6)

The method of claim 6, wherein the blending is performed using a mirroring with distance transform.

### Claim 12 (depends on 11)

The method of claim 11 wherein the mirroring comprises:
performing thresholding on the non-active area mask to generate a binary image content mask;
applying morphological smoothing to the binary image content mask;
calculating a distance transform of the binary image content mask;
using the distance transform, for each pixel in the non-active area, identifying a nearest pixel in an image content area to the each pixel;
generating an index map of nearest pixels, and
generating a blended image using the image and the index map of the nearest pixels.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
filtering the blended image to generate a filtered blended image.

### Claim 14 (depends on 13)

The method of claim 13, wherein the filtering comprises edge-preserving filtering.

### Claim 15 (depends on 13)

The method of claim 13, further comprising performing a weighted addition of the image and the filtered blended image.

### Claim 16 (depends on 15)

The method of claim 15, wherein the weighted addition is performed based on the non-active area probability map.

### Claim 17 (depends on 1)

The method of claim 1, wherein the image comprises an active area and the non-active area, wherein the detected non-active area is an area having a constant pixel value with some noise and it is located at the boundary of the image, and the non-active area is a static area of the image.

### Claim 18 (depends on 1)

The method of claim 1, wherein the predefined area is located between the topmost (HL) rows and bottommost (HL) rows of the image, and/or between the leftmost (WL) columns and the rightmost (WL) columns of the image.

### Claim 19 (depends on 1)

The method of claim 1, wherein classifying the presence of the non-active area in the image, thereby generating the boundary detection score, is based on a maximum ratio of the pixels that belong to the peak on a minimum possible non-active areas at the predefined area.

### Claim 20 (depends on 1)

The method of claim 1, wherein the peak detection score and the boundary detection score are larger when there is more likelihood of the presence of the non-active area.

### Claim 21 (depends on 1)

The method of claim 1, wherein the total score is based on a minimum between the peak detection score and the boundary detection score.

### Claim 22 (depends on 1)

The method of claim 1, wherein the width of the peak is defined by bins on the two opposite sides of the peak where a value of the histogram drops to a ratio of a maximum value of the histogram.

### Claim 23 (depends on 1)

A non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 1.