# US 2018/0025474 A1 · Video processing method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025474 A1
- Document ID: US20180025474A1
- Status: Published application
- Kind: A1
- Application number: 15/629,702
- Filed: 2017-06-21
- Published: 2018-01-25
- Earliest claimed: 2016-07-20
- Assignees: Alibaba Group Holding Limited
- Applicants: Alibaba Group Holding Limited
- Inventors: Dawei MEI; Wenfei JIANG; Li PAN
- Web page: https://patentgrep.com/patent/US20180025474A1

## CPC

- G06T 5/70: Denoising; Smoothing
- G06T 5/20: using local operators
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments of the disclosure provide a video processing method and a video processing apparatus. The video processing method may include estimating noise of an input video to obtain noise level information of the input video; and selecting, according to the noise level information, a denoising scheme to process the input video. The denoising scheme may merge a plurality of denoising algorithms.

## Claims

### Claim 1

A video processing method, comprising:
estimating noise of an input video to obtain noise level information of the input video; and
selecting, according to the noise level information, a denoising scheme to process the input video.

### Claim 2 (depends on 1)

The method according to claim 1, wherein estimating noise of an input video to obtain noise level information of the input video further comprises:
acquiring original image data and corresponding filtered image data of the input video;
determining noise estimation information of the set of frames of the input video based on statistics information of the original image data and the corresponding filtered image data; and
obtaining, according to the noise estimation information of each frame of the set of frames, smoothed noise estimation information as the noise level information of the input video.

### Claim 3 (depends on 2)

The method according to claim 2, wherein determining noise estimation information of a set of frames of the input video based on statistics information of the original image data and the corresponding filtered image data further comprises:
performing a block variance calculation on the original image data and the corresponding filtered image data respectively, to obtain corresponding original statistics information and processing statistics information; and
comparing the original statistics information with the processing statistics information, to determine the noise estimation information of the set of frames.

### Claim 4 (depends on 1)

The method according to claim 1, wherein selecting, according to the noise level information, a denoising scheme to process the input video further comprises:
comparing the noise level information with a preset denoising threshold, to determine whether the input video is a video-to-be-denoised,
in response to the input video being not a video-to-be-denoised, the denoising scheme outputting the input video, and
in response to the input video being a video-to-be-denoised, the denoising scheme selecting a denoising algorithm of the video-to-be-denoised according to the noise level information.

### Claim 5 (depends on 4)

The method according to claim 4, wherein selecting a denoising algorithm of the video-to-be-denoised according to the noise level information further comprises:
determining a denoising parameter of the video-to-be-denoised according to the noise level information; and
determining the denoising algorithm of the video-to-be-denoised according to the denoising parameter.

### Claim 6 (depends on 5)

The method according to claim 5, wherein determining the denoising parameter of the video-to-be-denoised according to the noise level information further comprises:
determining the denoising parameter of the video-to-be-denoised according to the noise level information and encoder information.

### Claim 7 (depends on 4)

The method according to claim 4, wherein the denoising algorithm of the video-to-be-denoised further comprises:
denoising the video-to-be-denoised by an inter-frame filtering algorithm.

### Claim 8 (depends on 7)

The method according to claim 7, wherein denoising the video-to-be-denoised by an inter-frame filtering algorithm further comprises:
calculating a motion index between a current frame of the video-to-be-denoised and an adjacent frame of the current frame; and
denoising the video-to-be-denoised by the inter-frame filtering algorithm according to the motion index.

### Claim 9 (depends on 8)

The method according to claim 8, wherein, after denoising the video-to-be-denoised by an inter-frame filtering algorithm, the method further comprises:
denoising the video-to-be-denoised by a simple intra-frame filtering algorithm;
merging denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm; and
outputting the merged result.

### Claim 10 (depends on 9)

The method according to claim 9, wherein denoising the video-to-be-denoised by a simple intra-frame filtering algorithm further comprises:
calculating a smoothing index in the video-to-be-denoised; and
denoising the video-to-be-denoised by the simple intra-frame filtering algorithm according to the smoothing index.

### Claim 11 (depends on 10)

The method according to claim 10, wherein merging denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm further comprise:
determining denoising strength based on the smoothing index and the motion index; and
determining whether the denoising strength is greater than a screening threshold,
in response to the denoising strength being greater than or equal to a screening threshold, merging the denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm.

### Claim 12 (depends on 7)

The method according to claim 7, wherein, after denoising the video-to-be-denoised by an inter-frame filtering algorithm, the method further comprises:
denoising the video-to-be-denoised by a simple intra-frame filtering algorithm;
denoising the video-to-be-denoised by a complicated intra-frame filtering algorithm; and
merging denoising results of the inter-frame filtering algorithm, the simple intra-frame filtering algorithm, and the complicated intra-frame filtering algorithm.

### Claim 13 (depends on 12)

The method according to claim 12, wherein denoising the video-to-be-denoised by a simple intra-frame filtering algorithm further comprises:
calculating a smoothing index in the video-to-be-denoised; and
denoising the video-to-be-denoised by the simple intra-frame filtering algorithm according to the smoothing index.

### Claim 14 (depends on 12)

The method according to claim 12, wherein denoising the video-to-be-denoised by a complicated intra-frame filtering algorithm further comprises:
determining denoising strength based on the smoothing index and the motion index; and
determining whether the denoising strength is greater than a screening threshold,
in response to the denoising strength being not greater than a screening threshold, denoising the current frame by the complicated intra-frame filtering algorithm.

### Claim 15 (depends on 11)

The method according to claim 11, wherein, after determining whether the denoising strength is greater than a screening threshold, the method further comprises:
making statistics on point number information of the current frame after the complicated intra-frame filtering algorithm; and
adjusting the screening threshold based on the point number information and a system complexity parameter.

### Claim 16

A video processing apparatus, comprising:
a noise estimation device used for estimating noise of an input video to obtain noise level information of the input video; and
a processing device used for selecting, according to the noise level information, a denoising scheme to process the input video.

### Claim 17 (depends on 16)

The apparatus according to claim 16, wherein the noise estimation device comprises:
an acquiring unit used for acquiring original image data and corresponding filtered image data of the input video;
a determining unit used for determining noise estimation information of a set of frames of the input video based on statistics information of the original image data and the corresponding filtered image data; and
an updating unit used for obtaining, according to the noise estimation information of each frame of the set of frames, smoothed noise estimation information as the noise level information of the input video.

### Claim 18 (depends on 17)

The apparatus according to claim 17, wherein the determining unit is used for:
performing a block variance calculation on the original image data and the corresponding filtered image data respectively, to obtain corresponding original statistics information and processing statistics information; and
comparing the original statistics information with the processing statistics information, to determine the noise estimation information of the set of frames.

### Claim 19 (depends on 16)

The apparatus according to claim 16, wherein the processing device is used for:
comparing the noise level information with a preset denoising threshold, to determine whether the input video is a video-to-be-denoised,
in response the input video being not a video-to-be-denoised, the denoising scheme is further used for outputting the input video, and
in response the input video being a video-to-be-denoised, the denoising scheme is further used for selecting a denoising algorithm of the video-to-be-denoised according to the noise level information.

### Claim 20 (depends on 19)

The apparatus according to claim 19, wherein the processing device is used for:
determining a denoising parameter of the video-to-be-denoised according to the noise level information; and
determining the denoising algorithm of the video-to-be-denoised according to the denoising parameter.

### Claim 21 (depends on 20)

The apparatus according to claim 20, wherein the processing device is used for:
determining the denoising parameter of the video-to-be-denoised according to the noise level information and encoder information.

### Claim 22 (depends on 19)

The apparatus according to claim 19, wherein the processing device is used for:
denoising the video-to-be-denoised by an inter-frame filtering algorithm.

### Claim 23 (depends on 22)

The apparatus according to claim 22, wherein the processing device is used for:
calculating a motion index between a current frame of the video-to-be-denoised and an adjacent frame of the current frame; and
denoising the video-to-be-denoised by the inter-frame filtering algorithm according to the motion index.

### Claim 24 (depends on 23)

The apparatus according to claim 23, wherein after the video-to-be-denoised is denoised by an inter-frame filtering algorithm, the processing device is used for:
denoising the video-to-be-denoised by a simple intra-frame filtering algorithm;
merging denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm; and
outputting the merged result.

### Claim 25 (depends on 24)

The apparatus according to claim 24, wherein the processing device is used for:
calculating a smoothing index in the video-to-be-denoised; and
denoising the video-to-be-denoised by the simple intra-frame filtering algorithm according to the smoothing index.

### Claim 26 (depends on 25)

The apparatus according to claim 25, wherein the processing device is used for:
determining denoising strength based on the smoothing index and the motion index; and
determining whether the denoising strength is greater than a screening threshold,
in response the denoising strength being greater than or equal to a screening threshold, merging the denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm.

### Claim 27 (depends on 22)

The apparatus according to claim 22, wherein, after the video-to-be-denoised is denoised by an inter-frame filtering algorithm, the processing device is used for:
denoising the video-to-be-denoised by a simple intra-frame filtering algorithm;
denoising the video-to-be-denoised by a complicated intra-frame filtering algorithm; and
merging denoising results of the inter-frame filtering algorithm, the simple intra-frame filtering algorithm, and the complicated intra-frame filtering algorithm.

### Claim 28 (depends on 27)

The apparatus according to claim 27, wherein the processing device is used for:
calculating a smoothing index in the video-to-be-denoised; and
denoising the video-to-be-denoised by the simple intra-frame filtering algorithm according to the smoothing index.

### Claim 29 (depends on 27)

The apparatus according to claim 27, wherein the processing device is used for:
determining denoising strength based on the smoothing index and the motion index; and
determining whether the denoising strength is greater than a screening threshold,
in response the denoising strength is not greater than a screening threshold, denoising the current frame by the complicated intra-frame filtering algorithm.

### Claim 30 (depends on 26)

The apparatus according to claim 26, wherein, the processing device is used for:
making statistics on point number information of the current frame after complicated intra-frame filtering algorithm; and
adjusting the screening threshold based on the point number information and a system complexity parameter.

### Claim 31

A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a video processing apparatus to cause the video processing apparatus to perform a video processing method, the method comprising:
estimating noise of an input video to obtain noise level information of the input video; and
selecting, according to the noise level information, a denoising scheme to process the input video.

### Claim 32 (depends on 31)

The non-transitory computer readable medium according to claim 31, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform estimating noise of an input video to obtain noise level information of the input video by:
acquiring original image data and corresponding filtered image data of the input video;
determining noise estimation information of a set of frames of the input video based on statistics information of the original image data and the corresponding filtered image data; and
obtaining, according to the noise estimation information of each frame of the set of frames, smoothed noise estimation information as the noise level information of the input video.

### Claim 33 (depends on 32)

The non-transitory computer readable medium according to claim 32, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform determining noise estimation information of a set of frames of the input video based on statistics information of the original image data and the corresponding filtered image data by:
performing a block variance calculation on the original image data and the corresponding filtered image data respectively, to obtain corresponding original statistics information and processing statistics information; and
comparing the original statistics information with the processing statistics information, to determine the noise estimation information of the set of frames.

### Claim 34 (depends on 31)

The non-transitory computer readable medium according to claim 31, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform selecting, according to the noise level information, a denoising scheme to process the input video by:
comparing the noise level information with a preset denoising threshold, to determine whether the input video is a video-to-be-denoised,
in response to the input video being not a video-to-be-denoised, the denoising scheme outputting the input video, and
in response to the input video being a video-to-be-denoised, the denoising scheme selecting a denoising algorithm of the video-to-be-denoised according to the noise level information.

### Claim 35 (depends on 34)

The non-transitory computer readable medium according to claim 34, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform selecting a denoising algorithm of the video-to-be-denoised according to the noise level information by:
determining a denoising parameter of the video-to-be-denoised according to the noise level information; and
determining the denoising algorithm of the video-to-be-denoised according to the denoising parameter.

### Claim 36 (depends on 35)

The non-transitory computer readable medium according to claim 35, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform determining a denoising parameter of the video-to-be-denoised according to the noise level information by:
determining the denoising parameter of the video-to-be-denoised according to the noise level information and encoder information.

### Claim 37 (depends on 34)

The non-transitory computer readable medium according to claim 34, wherein the denoising algorithm of the video-to-be-denoised comprises:
denoising the video-to-be-denoised by an inter-frame filtering algorithm.

### Claim 38 (depends on 37)

The non-transitory computer readable medium according to claim 37, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform denoising the video-to-be-denoised by an inter-frame filtering algorithm by:
calculating a motion index between a current frame of the video-to-be-denoised and an adjacent frame of the current frame; and
denoising the video-to-be-denoised by the inter-frame filtering algorithm according to the motion index.

### Claim 39 (depends on 38)

The non-transitory computer readable medium according to claim 38, wherein, after denoising the video-to-be-denoised by an inter-frame filtering algorithm, the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to further perform the method by:
denoising the video-to-be-denoised by a simple intra-frame filtering algorithm;
merging denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm; and
outputting the merged result.

### Claim 40 (depends on 39)

The non-transitory computer readable medium according to claim 39, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform denoising the video-to-be-denoised by a simple intra-frame filtering algorithm by:
calculating a smoothing index in the video-to-be-denoised; and
denoising the video-to-be-denoised by the simple intra-frame filtering algorithm according to the smoothing index.

### Claim 41 (depends on 40)

The non-transitory computer readable medium according to claim 40, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform merging denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm by:
determining denoising strength based on the smoothing index and the motion index; and
determining whether the denoising strength is greater than a screening threshold,
in response to the denoising strength being greater than or equal to a screening threshold, merging the denoising results of the inter-frame filtering algorithm and the simple intra-frame filtering algorithm.

### Claim 42 (depends on 37)

The non-transitory computer readable medium according to claim 37, wherein, after denoising the video-to-be-denoised by an inter-frame filtering algorithm, the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to further perform the method by:
denoising the video-to-be-denoised by a simple intra-frame filtering algorithm;
denoising the video-to-be-denoised by a complicated intra-frame filtering algorithm; and
merging denoising results of the inter-frame filtering algorithm, the simple intra-frame filtering algorithm, and the complicated intra-frame filtering algorithm.

### Claim 43 (depends on 42)

The non-transitory computer readable medium according to claim 42, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform denoising the video-to-be-denoised by a simple intra-frame filtering algorithm by:
calculating a smoothing index in the video-to-be-denoised; and
denoising the video-to-be-denoised by the simple intra-frame filtering algorithm according to the smoothing index.

### Claim 44 (depends on 42)

The non-transitory computer readable medium according to claim 42, wherein the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to perform denoising the video-to-be-denoised by a complicated intra-frame filtering algorithm by:
determining denoising strength based on the smoothing index and the motion index; and
determining whether the denoising strength is greater than a screening threshold,
in response to the denoising strength being not greater than a screening threshold, denoising the current frame by the complicated intra-frame filtering algorithm.

### Claim 45 (depends on 41)

The non-transitory computer readable medium according to claim 41, wherein, after determining whether the denoising strength is greater than a screening threshold, the set of instructions is executable by the at least one processor of the video processing apparatus to cause the video processing apparatus to further perform the method by:
making statistics on point number information of the current frame after complicated intra-frame filtering algorithm; and
adjusting the screening threshold based on the point number information and a system complexity parameter.