# US 2010/0165204 A1 · System and process for image rescaling using adaptive interpolation kernel with sharpness and overshoot control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0165204 A1
- Document ID: US20100165204A1
- Status: Published application
- Kind: A1
- Application number: 12/655,254
- Filed: 2009-12-23
- Published: 2010-07-01
- Earliest claimed: 2008-12-31
- Assignees: STMicroelectronics Asia Pacific Pte Ltd; STMicroelectronics S.r.l.
- Inventors: Yong Huang; Lucas Hui; Haiyun Wang; Fritz Lebowsky
- Web page: https://patentgrep.com/patent/US20100165204A1

## CPC

- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- G06T 5/20: using local operators
- G06T 5/70: Denoising; Smoothing
- G06T 5/73: Deblurring; Sharpening
- G06T 2200/12: involving antialiasing
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20192: Edge enhancement; Edge preservation
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Provided are a digital video rescaling system, a method of rescaling video images, and a chip comprising a computer executable medium embedded therein computer executable instructions for rescaling video images.

## Claims

### Claim 1

A digital video rescaling system comprising:
an input module configured to receive an input image data;
an adaptive spline interpolation kernel coupled to the input module, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel based on the input image data;
a sharpness control module coupled to the adaptive spline interpolation kernel, wherein the sharpness control module is configured to generate sharpness control parameters to control the adaptive spline interpolation kernel; and
an overshoot control module coupled to the adaptive spline interpolation kernel, wherein the overshoot control module is configured to generate overshoot control parameters to control the adaptive spline interpolation kernel.

### Claim 2 (depends on 1)

The digital video rescaling system of claim 1, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel y1(x) based on the following equation:
y
1

(
x
)
=
∑
n
=
A
,
B
,
C
,
D

y
n
*
f
n

(
s
,
Kshp
)
,
wherein yA, yB, yC and yD are four support pixels from input image data, wherein s is the distance from interpolation position to the position of the support pixel yb, wherein Kshp is a global sharpness control parameter generated by the sharpness control module, and wherein fn(s,Kshp) (n=A, B, C, D) are four control synthesis functions which can be expressed by the following equations:
fA(s,Kshp)=Kshp*(−s3+2*s2−s)/2;
fB(s,Kshp)=(3*Kshp*s3−5*Kshp*s2+2*(Kshp−1)*s+2)/2;
fC(s,Kshp)=(−3*Kshp*s3+4*Kshp*s2+(2−(Kshp)*s)/2; and
fD(s,Kshp)=Kshp*(s3−s2)/2.

### Claim 3 (depends on 2)

The digital video rescaling system of claim 2, wherein the overshoot control module comprises:
a max-min calculation unit configured to receive the data of pixels yB and yC, and configured to calculate the maximum value and the minimum value of the support pixels yB and yc;
a comparator coupled to the max-min calculation unit and adaptive spline interpolation kernel, wherein the comparator is configured to compare the maximum value and the minimum value of the support pixels yB and yc with the interpolated pixel y1(x) to generate an output y2(x) according the following equation:
y
2

(
x
)
=
{
max

(
y
B
,
y
C
)
if


(
y
1

(
x
)
>
max

(
y
B
,
y
C
)
)
min

(
y
B
,
y
C
)
if


(
y
1

(
x
)
<
min

(
y
B
,
y
C
)
)
y
1

(
x
)
else
;
a subtractor coupled to the max-min calculation unit and the comparator, configured to subtract the outputs y1(x) and y2(x), thereby generating an output (y1(x)−y2(x)); and
a multiplier coupled to the subtractor, wherein the multiplier is configured to multiply the output (y1(x)−y2(x)) by an overshoot control parameter Kovs to generate an output Kovs*(y1(x)−y2(x)); and
whereby the final output y(x)=Kovs*(y1(x)−y2(x))+y2(x) is based on the addition of y2(x) to Kovs*(y1(x)−y2(x)).

### Claim 4 (depends on 1)

The digital video rescaling system of claim 1 further comprising a local sharpness control module to adapt the sharpness control module to the local image content of the input image data, wherein the local sharpness control module comprises:
a local gradient estimation module coupled to the input module, wherein the local gradient estimation module is configured to generate a local gradient magnitude based on the input image data;
a noise level estimator coupled to the input module, wherein the noise level estimator is configured to generate a noise level based on the input image data; and
a local sharpness control parameter function module configured to control the sharpness control module using a transfer function based on the local gradient magnitude and the noise level.

### Claim 5 (depends on 4)

The digital video rescaling system of claim 4, wherein the transfer function of the local sharpness control parameter function module can be expressed as:
Kshp(Gn)=AKshp(Gn)+GKshp,
wherein GKshp is a fixed value that is defined by a global sharpness control parameter Kshp, AKshp(Gn) is a variable value that is defined by the control parameters max K, min K, K1, K2, K3, and Sigma, wherein Sigma is the noise level, wherein Gn is the local gradient magnitude; and
wherein AKshp(Gn) can be expressed by a transfer function according to the following equation:
AKshp

(
G
n
)
=
{

min


K

*
G
n
Sigma
+
min


K
if


(
G
n
≤
Sigma
)
max


K
*
(
G
n
-
Sigma
)
K


1
-
Sigma
if


(
Sigma
<
G
n
≤
K


1
)
max


K
if


(
K


1
<
G
n
≤
K


2
)
max


K
*
(
K


3
-
G
n
)
K


3
-
K


2
if


(
K


2
<
G
n
≤
K


3
)
0
else
.

### Claim 6 (depends on 1)

The digital video rescaling system of claim 1 further comprising an anti-aliasing filter coupled between the input module and the adaptive spline interpolation kernel, wherein the anti-aliasing filter comprises a set of low pass filters that are configured to reduce aliasing artifacts during image downscaling.

### Claim 7 (depends on 6)

The digital video rescaling system of claim 6 further comprising a switch device coupled between the input module and the anti-aliasing filter, wherein the switch device is configured to activate the anti-aliasing filter during downscaling, and wherein the switch device is configured to deactivate the anti-aliasing filter during upscaling.

### Claim 8

A method of rescaling video images including:
receiving data corresponding to an input image;
calculating interpolated pixel values based on the input image data by using an adaptive spline interpolation kernel;
generating sharpness control parameters to control the adaptive spline interpolation kernel by using a sharpness control module; and
generating overshoot control parameters to control the adaptive spline interpolation kernel by using an overshoot control module.

### Claim 9 (depends on 8)

The method of claim 8, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel y1(x) based on the following equation:
y
1

(
x
)
=
∑
n
=
A
,
B
,
C
,
D

y
n
*
f
n

(
s
,
Kshp
)
,
wherein yA, yB, yC and yD are four support pixels from input image data, wherein s is the distance from interpolation position to the position of the support pixel yb, wherein Kshp is a global sharpness control parameter generated by the sharpness control module, and wherein fn(s,Kshp) (n=A, B, C, D) are four control synthesis functions which can be expressed by the following equations:
fA(s,Kshp)=Kshp*(−s3+2*s2−s)/2;
fB(s,Kshp)=(3*Kshp*s3−5*Kshp*s2+2*(Kshp−1)*s+2)/2;
fC(s,Kshp)=(−3*Kshp*s3+4*Kshp*s2+(2−(Kshp)*s)/2; and
fD(s,Kshp)=Kshp*(s3−s2)/2.

### Claim 10 (depends on 9)

The method of claim 9, wherein the operation of generating overshoot control parameters to control the adaptive spline interpolation kernel comprises:
calculating a maximum value and a minimum value of the support pixels yB and yc;
comparing the maximum value and the minimum value of the support pixels yB and yC with the interpolated pixel y1(x) to generate an output y2(x) according the following equation:
y
2

(
x
)
=
{
max

(
y
B
,
y
C
)
if


(
y
1

(
x
)
>
max

(
y
B
,
y
C
)
)
min

(
y
B
,
y
C
)
if


(
y
1

(
x
)
<
min

(
y
B
,
y
C
)
)
y
1

(
x
)
else
;
subtracting the outputs y1(x) and y2(x) to generate an output (y1(x)−y2(x));
multiplying the output (y1(x)−y2(x)) by an overshoot control parameter Kovs to generate an output Kovs*(y1(x)−y2(x)); and
generating the final output y(x)=Kovs*(y1(x)−y2(x))+y2(x) by adding y2(x) to the output Kovs*(y1(x)−y2(x)).

### Claim 11 (depends on 8)

The method of claim 8 further comprising:
generating a local gradient magnitude based on the input image data;
generating a noise level based on the input image data; and
controlling the sharpness control module by using a transfer function based on the local gradient magnitude and the noise level.

### Claim 12 (depends on 11)

The method of claim 11, wherein the transfer function can be expressed as:
Kshp(Gn)=AKshp(Gn)+GKshp,
wherein GKshp is a fixed value that is defined by a global sharpness control parameter Kshp, AKshp(Gn) is a variable value that is defined by the control parameters max K, min K, K1, K2, K3, and Sigma, wherein Sigma is the noise level, wherein Gn is the local gradient magnitude, and
wherein AKshp(Gn) can be expressed by a transfer function:
AKshp

(
G
n
)
=
{

min


K

*
G
n
Sigma
+
min


K
if


(
G
n
≤
Sigma
)
max


K
*
(
G
n
-
Sigma
)
K


1
-
Sigma
if


(
Sigma
<
G
n
≤
K


1
)
max


K
if


(
K


1
<
G
n
≤
K


2
)
max


K
*
(
K


3
-
G
n
)
K


3
-
K


2
if


(
K


2
<
G
n
≤
K


3
)
0
else
.

### Claim 13 (depends on 8)

The method of claim 8 further comprising:
selecting a set of filter coefficients from a low pass filter bank according to a downscaling factor; and
applying the set of filter coefficients on the input image data to reduce anti-aliasing effects during downscaling.

### Claim 14

A chip comprising a computer executable medium embedded therein computer executable instructions for rescaling video images, wherein the instructions comprise:
receiving data corresponding to an input image;
calculating interpolated pixel values based on the input image data by using an adaptive spline interpolation kernel;
generating sharpness control parameters to control the adaptive spline interpolation kernel by using a sharpness control module; and
generating overshoot control parameters to control the adaptive spline interpolation kernel by using an overshoot control module.

### Claim 15 (depends on 14)

The chip of claim 14, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel y1(x) based on the following equation:
y
1

(
x
)
=
∑
n
=
A
,
B
,
C
,
D

y
n
*
f
n

(
s
,
Kshp
)
,
wherein yA, yB, yC and yD) are four support pixels from input image data, wherein s is the distance from interpolation position to the position of the support pixel yb, wherein Kshp is a global sharpness control parameter generated by the sharpness control module, and wherein fn(s,Kshp) (n=A, B, C, D) are four control synthesis functions which can be expressed by the following equations:
fA(s,Kshp)=Kshp*(−s3+2*s2−s)/2;
fB(s,Kshp)=(3*Kshp*s3−5*Kshp*s2+2*(Kshp−1)*s+2)/2;
fC(s,Kshp)=(−3*Kshp*s3+4*Kshp*s2+(2−(Kshp)*s)/2; and
fD(s,Kshp)=Kshp*(s3−s2)/2.

### Claim 16 (depends on 15)

The chip of claim 15, wherein the operation of generating overshoot control parameters to control the adaptive spline interpolation kernel comprises:
calculating a maximum value and a minimum value of the support pixels yB and yC;
comparing the maximum value and the minimum value of the support pixels yB and yC with the interpolated pixel y1(x) to generate an output y2(x) according the following equation:
y
2

(
x
)
=
{
max

(
y
B
,
y
C
)
if


(
y
1

(
x
)
>
max

(
y
B
,
y
C
)
)
min

(
y
B
,
y
C
)
if


(
y
1

(
x
)
<
min

(
y
B
,
y
C
)
)
y
1

(
x
)
else
;
subtracting the outputs y1(x) and y2(x) to generate an output (y1(x)−y2(x));
multiplying the output (y1(x)−y2(x)) by an overshoot control parameter Kovs to generate an output Kovs*(y1(x)−y2(x)); and
generating the final output y(x)=Kovs*(y1(x)−y2(x))+y2(x) by adding y2(x) to the output Kovs*(y1(x)−y2(x)).

### Claim 17 (depends on 14)

The chip of claim 14, wherein the instructions further comprise:
generating a local gradient magnitude based on the input image data;
generating a noise level based on the input image data; and
controlling the sharpness control module by using a transfer function based on the local gradient magnitude and the noise level.

### Claim 18 (depends on 17)

The chip of claim 17, wherein the transfer function can be expressed as:
Kshp(Gn)=AKshp(Gn)+GKshp,
wherein GKshp is a fixed value that is defined by a global sharpness control parameter Kshp, AKshp(Gn) is a variable value that is defined by the control parameters max K, min K, K1, K2, K3, and Sigma, wherein Sigma is the noise level, wherein Gn is the local gradient magnitude, and
wherein AKshp(Gn) can be expressed by a transfer function:
AKshp

(
G
n
)
=
{

min


K

*
G
n
Sigma
+
min


K
if


(
G
n
≤
Sigma
)
max


K
*
(
G
n
-
Sigma
)
K


1
-
Sigma
if


(
Sigma
<
G
n
≤
K


1
)
max


K
if


(
K


1
<
G
n
≤
K


2
)
max


K
*
(
K


3
-
G
n
)
K


3
-
K


2
if


(
K


2
<
G
n
≤
K


3
)
0
else
.

### Claim 19 (depends on 14)

The chip of claim 14, wherein the instructions further comprise:
selecting a set of filter coefficients from a low pass filter bank according to a downscaling factor; and
applying the set of filter coefficients on the input image data to reduce anti-aliasing effects during downscaling.

### Claim 20 (depends on 19)

The chip of claim 19, wherein the instructions further comprise:
deactivating the set of filter coefficients on the input image data during upscaling.
