# US 2021/0264566 A1 · Method and system for image scaling and enhancement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0264566 A1
- Document ID: US20210264566A1
- Status: Published application
- Kind: A1
- Application number: 16/924,215
- Filed: 2020-07-09
- Published: 2021-08-26
- Earliest claimed: 2020-02-20
- Assignees: Realtek Semiconductor Corp.
- Applicants: Realtek Semiconductor Corp.
- Inventors: Chia-Wei Yu; Yi-Ting Bao; Yen-Lin Chen; Shang-Yen Lin
- Web page: https://patentgrep.com/patent/US20210264566A1

## CPC

- G06T 3/4046: using neural networks
- G06T 3/4076: using the original low-resolution images to iteratively correct the high-resolution images
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/90: Determination of colour characteristics
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system for image scaling and enhancement is provided. The system includes a scaling processing unit, a deep-learning residue network unit and a combination unit. The filter scaling processing unit is configured to upscale a low-resolution image to output a high-resolution image. The deep-learning residue network unit is operated based on a deep-learning result, and configured to output a high-resolution residue image corresponding to the low-resolution image. The combination unit is configured to adjust the high-resolution residue image according to a weighting factor and combine an adjusted high-resolution residue image and the high-resolution image, in order to output an enhanced image, wherein the weighting factor is different from a reference weighting factor being used in a deep-learning procedure for training the deep-learning residue network unit.

## Claims

### Claim 1

An image scaling and enhancement system, comprising:
a scaling processing unit, configured to upscale a low-resolution image to output a high-resolution image;
a deep-learning residue network unit, operated based on a deep-learning result, configured to output a high-resolution residue image corresponding to the low-resolution image; and
a combination processing unit, coupled to the scaling processing unit and the deep-learning residue network unit, configured to adjust the high-resolution residue image according to a weighting factor and combine an adjusted high-resolution residue image and the high-resolution image in order to generate an enhanced image, wherein the weighting factor is different from a reference weighting factor being used in a deep-learning procedure for training the deep-learning residue network unit.

### Claim 2 (depends on 1)

The system of claim 1, wherein the weighting factor is an user-defined fixed value or a value determined based on content characteristics of the low-resolution image.

### Claim 3 (depends on 1)

The system of claim 1, further comprising:
a weighting factor determination unit, coupled to the combination processing unit, configured to determine the weighting factor.

### Claim 4 (depends on 3)

The system of claim 3, wherein the weighting factor determination unit is configured to determine a weighting factor for each pixel according to the low-resolution image.

### Claim 5 (depends on 3)

The system of claim 3, wherein the weighting factor determination unit is configured to determine a weighting factor for each pixel according to the high-resolution image.

### Claim 6 (depends on 3)

The system of claim 3, wherein the weighting factor determination unit is configured to determine the weighting factor according to high-frequency characteristics or boundary strength of pixels around a specific pixel of one of the low-resolution image and the high-resolution image.

### Claim 7 (depends on 3)

The system of claim 3, wherein the weighting factor determination unit is configured to determine the weighting factor according to color characteristics of pixels around a specific pixel of one of the low-resolution image and the high-resolution images.

### Claim 8 (depends on 1)

The system of claim 1, wherein the deep-learning residue network unit comprises an arbitrary number of layers or structures.

### Claim 9

An image scaling and enhancement method, comprising:
upscaling a low-resolution image to output a high-resolution image;
based on a deep-learning result, outputting a high-resolution residue image corresponding to the low-resolution image; and
adjusting the high-resolution residue image according to a weighting factor and combining an adjusted high-resolution residue image and the high-resolution image in order to generate an enhanced image, wherein the weighting factor is different from a reference weighting factor being used in obtaining the deep-learning result.

### Claim 10 (depends on 9)

The method of claim 9, wherein the weighting factor is an user-defined fixed value or a value determined based on content characteristics of the low-resolution image.

### Claim 11 (depends on 9)

The method of claim 9, further comprising:
determining a weighting factor for each pixel according to one of the low-resolution image and the high-resolution image.

### Claim 12 (depends on 11)

The method of claim 11, wherein the step of determining the weighting factor for each pixel comprises:
determining the weighting factor according to high-frequency characteristics or boundary strength of pixels around a specific pixel of one of the low-resolution image and the high-resolution image.

### Claim 13 (depends on 11)

The method of claim 11, wherein the step of determining the weighting factor for each pixel comprises:
determining the weighting factor according to color characteristics of pixels around a specific pixel of one of the low-resolution image and the high-resolution image.
