# US 8,279,263 B2 · Mapping psycho-visual characteristics in measuring sharpness feature and blurring artifacts in video streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,263 B2
- Document ID: US8279263B2
- Status: Granted
- Kind: B2
- Application number: 12/566,633
- Filed: 2009-09-24
- Published: 2011-03-24
- Granted: 2012-10-02
- Earliest claimed: 2009-09-24
- Assignees: Microsoft Corporation
- Inventors: Walid S. I. Ali; Rony Ferzli
- Web page: https://patentgrep.com/patent/US8279263B2

## CPC

- H04N 7/15: Conference systems
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 21/6377: directed to server {(one-way streaming services wherein the source is controlled by the destination H04L65/613)}
- H04N 21/658: Transmission by the client directed to the server
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A block-based, no-reference sharpness metric is provided taking advantage of Human Visual System (HVS) characteristics. Texture and smooth region blocks are excluded in computing the metric since sharpness is perceived mostly around edges. Overall sharpness metric is computed by pooling simulated combination of information in human brain employing a logistic function to replicate the behavior of HVS.

## Claims

### Claim 1

A method to be executed at least in part in a computing device for providing a sharpness metric associated with received video quality in a video conference system, the method comprising:
receiving a decoded video frame at a post-processing module;
dividing the frame into blocks;
determining edge blocks by:
determining a number of edge pixels in each block;
if the number of edge pixels in a block is below a predefined threshold, designating the block a smooth block;
if the number of edge pixels in another block exceeds the predefined threshold, determining numbers of edge pixels in neighboring blocks of the other block, and
if the numbers of edge pixels in all neighboring blocks exceed the predefined threshold, designating the other block a texture block, else
designating the other block an edge block;
computing local sharpness metrics for each of the edge blocks based on a kurtosis and a motion vector for each respective edge block; and
computing an overall sharpness metric for the frame by pooling the local sharpness metrics over the entire frame.

### Claim 2 (depends on 1)

The method of claim 1, wherein the smooth blocks and the texture blocks of the frame are excluded from the overall sharpness metric computation.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
computing an edge map of the frame employing Sobel operator.

### Claim 4 (depends on 1)

The method of claim 1, wherein the local sharpness metric for each block is computed by:
determining discrete cosine transform (DCT) coefficients and the kurtosis for each block, wherein the DCT coefficients are handled as a probability density function;
determining a motion vector length for each block; and
computing the local sharpness metric by multiplying the kurtosis and the motion vector length for each block.

### Claim 5 (depends on 1)

The method of claim 1, wherein pooling the local sharpness metrics over the entire frame includes summing the local sharpness metrics of the frame by applying a non-linear model to replicate Human Visual System (HVS).

### Claim 6 (depends on 5)

The method of claim 5, wherein the non-linear model is an exponential model and defines an upper and a lower limit for the overall sharpness metric.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
employing the overall sharpness metric to adjust at least one operational parameter of the video conference system for automatically reducing blockiness artifacts in received video.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
combining the overall sharpness metric with at least one from a set of a blockiness metric, a jerkiness metric, and a dropped frame number to adjust at least one operational parameter of the video conference system for automatically enhancing received video quality.

### Claim 9 (depends on 8)

The method of claim 8, wherein the at least one operational parameter is adjusted at one of: a pre-processing module, an encoder, a decoder, and the post-processing module of the video conference system.

### Claim 10

A computing device capable of providing video conferencing services, the computing device comprising:
a video capture device;
a pre-processing module;
an encoder;
a decoder;
a post-processing module configured to:
receive a decoded video frame;
divide the frame into blocks;
determine smooth, texture, and edge blocks;
exclude smooth and texture blocks;
compute local sharpness metrics for each of the edge blocks employing an activity detection algorithm; and
compute an overall sharpness metric for the frame by pooling the local sharpness metrics over the entire frame, wherein the overall sharpness metric is used to adjust at least one operational parameter of the video conference system for automatically enhance received video; and
a display device for displaying the enhanced video.

### Claim 11 (depends on 10)

The computing device of claim 10, wherein the post-processing module is further configured to determine the edge blocks by:
determining an edge map corresponding to the frame;
determining a number of edge pixels in each of the blocks; and
designating the smooth, texture, and edge blocks based on a number of the edge pixels in each of the blocks and in neighboring blocks.

### Claim 12 (depends on 10)

The computing device of claim 10, wherein the post-processing module is further configured to compute the local sharpness metric for each block by:
compute a kurtosis “K” for each edge block based on DCT coefficients for each edge block;
compute a motion vector length “W” for each edge block based on W=|1−SAD|, where SAD represents a sum of absolute differences; and
compute the local sharpness metric “B” by multiplying the kurtosis and the motion vector length for each block as B =W*K.

### Claim 13 (depends on 10)

The computing device of claim 10, wherein pooling the local sharpness metrics over the entire frame includes applying a non-linear logistic function to a combination of the local sharpness metrics of the edge blocks within the frame such that extreme values of overall sharpness metric are suppressed.

### Claim 14 (depends on 10)

The computing device claim 10, wherein the overall sharpness metric is employed to perform one of:
instruct the encoder to increase a transmit bit rate in response to the metric being below a predefined threshold; and
instruct the post-processing module to one of increase a filtering strength and decrease a filter mask size in response to the metric being below the predefined threshold.

### Claim 15 (depends on 10)

The computing device of claim 10, wherein a Multipoint Control Unit (MCU) of the video conference system is employed to receive the overall sharpness metric from a receiving participant and instruct a transmitting participant to adjust the at least one operational parameter for enhancing the received video.

### Claim 16

A computer-readable memory device having instructions stored thereon for providing video conferencing services the instructions comprising:
receiving a decoded video frame at a post-processing module;
dividing the frame into blocks;
determining edge blocks by excluding smooth and texture blocks based on a number of the edge pixels in each of the blocks and in neighboring blocks;
computing local sharpness metrics for each of the edge blocks based on a product of a kurtosis and a motion vector for each respective edge block;
computing an overall sharpness metric for the frame by pooling the local sharpness metrics over the entire frame and applying an exponential logistic function; and
combining the overall sharpness metric with at least one from a set of: a blockiness metric, a jerkiness metric, and a dropped frame number to adjust at least one operational parameter of the video conference system for automatically enhancing received video quality.

### Claim 17 (depends on 16)

The computer-readable memory device of claim 16, wherein the overall sharpness metric “S” is computed as:
S
=
(
∑
i
=
1
R
⁢

B
⁡
(
i
)

β
)
1
β
,
where “R” is a total number of edge blocks within the frame, B(i) represents the local sharpness metric for each edge block, and β is a predefined constant.

### Claim 18 (depends on 16)

The computer-readable memory device of claim 16, wherein the non-linear logistic function is applied to the overall sharpness metric “S′” as:
S
′
=
5
1
+
ⅇ
-
300
⁢
(
S
-
0.002
)
,
where S′ is the adjusted overall sharpness metric, and numerator “5” provides an upper limit value of five to the overall sharpness metric.

### Claim 19 (depends on 16)

The computer-readable memory device of claim 16, wherein the instructions further comprise:
decreasing a filter size mask automatically in response to the overall sharpness metric being below a predetermined threshold.
