# US 2010/0165207 A1 · Apparatus and method of motion detection for temporal mosquito noise reduction in video sequences

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0165207 A1
- Document ID: US20100165207A1
- Status: Published application
- Kind: A1
- Application number: 12/494,124
- Filed: 2009-06-29
- Published: 2010-07-01
- Earliest claimed: 2008-12-29
- Assignees: STMicroelectronics Asia Pacific Pte Ltd
- Inventors: Xiaoyun Deng; Patricia Chiang
- Web page: https://patentgrep.com/patent/US20100165207A1

## CPC

- H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)}
- H04N 5/213: Circuitry for suppressing or minimising impulsive noise (for suppressing or minimising disturbance in image signal generation H04N23/81)
- H04N 5/142: {Edging; Contouring}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Apparatus and methods of motion detection for mosquito noise reduction in video sequences are provided. In one aspect, a method of motion detection in a sequence of digital images classifies a pixel of a plurality of pixels of a current image frame represented by a digital video input signal as a motion or non-motion pixel. A motion value for the pixel is calculated based on the classification of the pixel. The motion value is mapped to a coefficient of a temporal filter based on a control curve. A digital video output signal is generated based on the coefficient.

## Claims

### Claim 1

A method of motion detection in a sequence of digital images, the method comprising:
classifying a pixel of a plurality of pixels of a current image frame represented by a digital video input signal as a motion or non-motion pixel;
calculating a motion value for the pixel based on the classification of the pixel;
mapping the motion value to a coefficient of a temporal filter based on a control curve; and
generating a digital video output signal based on the coefficient.

### Claim 2 (depends on 1)

The method of claim 1, wherein classifying a pixel of a plurality of pixels of a current frame to be a motion or non-motion pixel comprises performing thresholding and spatial connectivity checking for the pixel of the plurality of pixels of the image.

### Claim 3 (depends on 2)

The method of claim 2, wherein performing thresholding comprises performing signed or unsigned thresholding using an edge and texture adaptive threshold provided by an adaptive threshold calculator.

### Claim 4 (depends on 3)

The method of claim 3, wherein performing signed or unsigned thresholding using an edge and texture adaptive threshold provided by an adaptive threshold calculator comprises performing signed or unsigned thresholding using analyzed edge and texture statistics information in a block of surrounding pixels or a coded discrete cosine transform (DCT) block of pixels.

### Claim 5 (depends on 2)

The method of claim 2, wherein performing thresholding comprises:
calculating a signed or unsigned pixel difference value between a filtered previous frame and the current frame; and
comparing the signed or unsigned pixel difference value against a respective signed or unsigned motion threshold.

### Claim 6 (depends on 3)

The method of claim 3, wherein performing thresholding using an edge and texture adaptive threshold provided by an adaptive threshold calculator comprises:
analyzing edge and texture statistics information in a block of surrounding pixels or a discrete cosine transformed (DCT) block of pixels;
calculating a relative edge value of a pixel, the relative edge being a detected edge gradient normalized by a maximum edge gradient over the block of surrounding pixels; and
mapping the relative edge value of the pixel to a motion threshold according to a mapping curve.

### Claim 7 (depends on 6)

The method of claim 6, wherein mapping the relative edge value of the pixel to a motion threshold according to a mapping curve comprises:
setting motion threshold values to be high for pixels with low relative edge values;
setting the motion threshold values to be low for pixels with high relative edge values;
adjusting the motion threshold values for flat area pixels or texture area pixels having low relative edge values based on a strength of a dominant edge and an average texture level in the block of surrounding pixels; and
adjusting the motion threshold values for dominant edge pixels based on an estimated picture noise level.

### Claim 8 (depends on 2)

The method of claim 2, wherein performing spatial connectivity checking for the pixel of the plurality of pixels of the image comprises:
counting a number of pixels in adjacent two rows and adjacent two columns each as having a respective absolute value of a respective positive, negative or absolute pixel difference exceeding a positive, negative or absolute motion threshold, respectively; and
comparing the number of count to a predetermined count threshold to check the connectivity to determine if the pixel is a motion or non-motion pixel.

### Claim 9 (depends on 1)

The method of claim 1, wherein calculating a motion value for the pixel based on the classification of the pixel comprises calculating the motion value for the pixel using an absolute pixel difference value and a suppressed motion value based on respective luminance and chrominance components of a filtered previous frame and the current frame.

### Claim 10 (depends on 1)

The method of claim 1, wherein calculating a motion value for the pixel based on the classification of the pixel comprises:
selecting an absolute pixel difference for motion classified pixel; and
selecting a suppressed motion value for a non-motion classified pixel.

### Claim 11

An apparatus for temporal mosquito noise reduction in digital imaging, comprising:
a motion detector coupled to receive a first video input signal and a second video input signal, the motion detector configured to classify a pixel of a plurality of pixels of a current frame of an image as a motion or non-motion pixel based on the received first and second video input signals, calculate a motion value for the pixel based on the classification of the pixel, and generate a coefficient;
a temporal filter coupled to receive the coefficient from the motion detector to generate a video output signal; and
a frame delay unit coupled to receive the video output signal from the temporal filter to delay the video output signal and generate the second video input signal.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the motion detector comprises:
an edge and texture analyzer coupled to receive the first video signal and the second video signal to output a gradient magnitude of a current pixel, a maximum gradient magnitude of a block of surrounding pixels or a discrete cosine transformed (DCT) block of pixels, and an average gradient magnitude of the block of surrounding pixels or the DCT block of pixels;
an adaptive threshold calculator coupled to receive the gradient magnitude of the current pixel, the maximum gradient magnitude of the block of surrounding pixels or the DCT block of pixels, and the average gradient magnitude of the block of surrounding pixels or the DCT block of pixels to output a pixel-based motion threshold;
a pixel difference calculator coupled to receive the first and second video signals to output a difference of the first and second video signals;
a motion classifier coupled to receive the pixel-based motion threshold and the difference of the first and second video signals to output a pixel-based motion classification value;
a noise cancellation filter coupled to receive the first and second video signals to output a suppressed motion value;
a motion value calculator coupled to receive the difference of the first and second video signals, the pixel-based motion classification value, and the suppressed motion value to output a motion value of the pixel; and
a control curve mapping unit coupled to receive the motion value of the pixel to output the coefficient to the temporal filter.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the edge and texture analyzer comprises:
a first edge detector coupled to receive a luminance component of the first video signal to output a pixel gradient magnitude of a current frame;
a second edge detector coupled to receive a luminance component of the second video signal to output a pixel gradient magnitude of a filtered previous frame;
a maximum operator coupled to receive the pixel gradient magnitude of the current frame and the pixel gradient magnitude of the filtered previous frame to output the gradient magnitude of the current pixel representative of the greater of the current frame and the pixel gradient magnitude of the filtered previous frame;
a block memory buffer coupled to receive the gradient value to store and output pixel gradients of the block of surrounding pixels or the DCT block of pixels;
a maximum edge detector coupled to receive the pixel gradients of the block of surrounding pixels or the DCT block of pixels to output the maximum gradient magnitude of the block of surrounding pixels or the DCT block of pixels; and
an average gradient calculator coupled to receive the pixel gradients of the block of surrounding pixels or the DCT block of pixels to output the average gradient magnitude of the block of surrounding pixels or the DCT block of pixels.

### Claim 14 (depends on 12)

The apparatus of claim 12, wherein the adaptive threshold calculator comprises:
an adaptive threshold curve calculator coupled to receive a picture motion threshold and a relative gradient threshold to output the pixel-based motion threshold.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the adaptive threshold calculator further comprises:
a relative gradient calculator coupled to receive the gradient magnitude of the current pixel and the maximum gradient magnitude of the block or surrounding pixels or the DCT block of pixels to output a relative gradient related to a ratio of the gradient magnitude of the current pixel and the maximum gradient magnitude of the block or surrounding pixels or the DCT block of pixels, wherein the adaptive threshold curve calculator further receives the average gradient magnitude of the block or surrounding pixels or the DCT block of pixels, the relative gradient, and a rising factor of the motion threshold in addition to the picture motion threshold and the relative gradient threshold to output the pixel-based motion threshold.

### Claim 16 (depends on 12)

The apparatus of claim 12, wherein the pixel difference calculator comprises:
a first subtractor coupled to receive and subtract a luminance component of the first video signal and a luminance component of the second video signal to output a first signed difference value;
a second subtractor coupled to receive and subtract a chrominance U component of the first video signal and a chrominance U component of the second video signal to output a second signed difference value;
a third subtractor coupled to receive and subtract a chrominance V component of the first video signal and a chrominance V component of the second video signal to output a third signed difference value;
a first absolute operator coupled to receive the first signed difference value to provide a first absolute difference value;
a second absolute operator coupled to receive the second signed difference value to provide a second absolute difference value;
a third absolute operator coupled to receive the third signed difference value to provide a third absolute difference value; and
a comparison unit coupled to receive at least the first, second, and third signed difference values to output the difference of the first and second video signals.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the pixel difference calculator further comprises:
a first low pass filter coupled to receive the first absolute difference value in a window to provide a first filtered difference value of a center pixel;
a second low pass filter coupled to receive the second absolute difference value in the window to provide a second filtered difference value of the center pixel; and
a third low pass filter coupled to receive the third absolute difference value in the window to provide a third filtered difference value of the center pixel,
and wherein the comparator unit comprises:
a comparator coupled to receive at least the first, second, and third filtered difference values to output a selection decision; and
a switch coupled to receive at least the first, second, and third signed difference values and the selection decision to output the difference of the first and second video signals.

### Claim 18 (depends on 12)

The apparatus of claim 12, wherein the noise cancellation filter comprises:
a first subtractor coupled to receive and subtract a luminance component of the second video signal from a luminance component of the first video signal to output a first signed difference value;
a second subtractor coupled to receive and subtract a chrominance U component of the second video signal from a chrominance U component of the first video signal to output a second signed difference value;
a third subtractor coupled to receive and subtract a chrominance V component of the second video signal from a chrominance V component of the first video signal to output a third signed difference value;
a first low pass filter coupled to receive and filter the first signed difference values in a window to output a first signed suppressed difference value of a center pixel;
a second low pass filter coupled to receive and filter the second signed difference values in a window to output a second signed suppressed difference value of the center pixel;
a third low pass filter coupled to receive and filter the third signed difference values in a window to output a third signed suppressed difference value of the center pixel;
a first absolute operator coupled to receive the first signed suppressed difference value to provide a first absolute suppressed difference value;
a second absolute operator coupled to receive the second signed suppressed difference value to provide a second absolute suppressed difference value;
a third absolute operator coupled to receive the third signed suppressed difference value to provide a third absolute suppressed difference value; and
a comparison unit coupled to receive and compare at least the first, second, and third absolute suppressed difference values to output one of the received first, second, and third absolute suppressed difference values having a maximum magnitude as a suppressed difference of the first and second video signals.

### Claim 19 (depends on 12)

The apparatus of claim 12, wherein the motion value calculator comprises:
an absolute operator coupled to receive difference of the first and second video signals to output an absolute value of the difference of the first and second video signals; and
a switch coupled to receive a motion classification decision, the suppressed motion value and a value related to the absolute value of the difference of the first and second video signals to output a value related to the motion value of the pixel based on the motion classification decision.

### Claim 20 (depends on 12)

The apparatus of claim 12, wherein the motion value calculator further comprises:
a motion value enhancer coupled to receive an absolute value of a difference of the first and second video signals to output an enhanced difference value of the first and second video signals; and
a low pass filter coupled to receive a motion value of the pixel based on the motion classification decision to output a post-processed value related to the motion value of the pixel.

### Claim 21

A digital image noise reduction device, comprising:
a filter unit coupled to receive a video input signal and a delayed video signal to output a filtered video output signal, the filter unit including:
a motion detector and temporal filter unit; and
a spatial filter unit coupled to the motion detector and temporal filter unit; and
a frame delay unit coupled to receive the filtered video output signal to output the delayed video signal.

### Claim 22 (depends on 21)

The device of claim 21, wherein the filter unit further comprises:
a noise level estimator coupled to receive the video input signal to output a picture noise level to the motion detector and temporal filter unit; and
a fader coupled to receive outputs of the motion detector and temporal filter unit and the spatial filter unit to output the filtered video output signal.

### Claim 23

A digital image noise reduction device, comprising:
a first temporal filter unit coupled to receive a signal related to a video input signal to output a first temporally filtered signal, the first temporal filter unit including:
a first frame delay unit coupled to receive the signal related to the video input signal to output a delayed video input signal; and
a first motion detector and temporal filter unit coupled to receive the video input signal and the delayed video input signal to output the first temporally filtered signal;
a second temporal filter unit coupled to receive a signal related to the first temporally filtered signal to output a second temporally filtered signal, the second temporal filter unit including:
a second frame delay unit coupled to receive a video output signal to output a delayed video output signal; and
a second motion detector and temporal filter unit coupled to receive the delayed video output signal and the signal related to the first temporally filtered signal to output the second temporally filtered signal; and
a spatial filter coupled to receive a signal to output a spatially filtered signal, the spatial filter being coupled between the video input signal and first temporal filter unit, between the first temporal filter unit and the second temporal filter unit, or between an output of the second motion detector and temporal filter unit and an input of the second frame delay unit.
