# US 12,182,968 B2 · Image processing method, apparatus, and device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,968 B2
- Document ID: US12182968B2
- Status: Granted
- Kind: B2
- Application number: 17/574,185
- Filed: 2022-01-12
- Published: 2022-05-05
- Granted: 2024-12-31
- Earliest claimed: 2019-07-12
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Huan Lin; Jie Ji; Jiangzheng Wu
- Web page: https://patentgrep.com/patent/US12182968B2

## CPC

- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 3/4069: by subpixel displacements
- G06T 3/4046: using neural networks
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20024: Filtering details
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing method, where, in the method, a special image filter is generated, and super-resolution is performed on an image based on the image filter. The image filter includes filter parameters corresponding to each pixel in an image that requires super-resolution processing, and pixels with different texture features correspond to different filter parameters. The image super-resolution method, apparatus, device, and the like may be applied to various scenarios such as a video, a game, and photographing.

## Claims

### Claim 1

An image processing method comprising:
obtaining a first image comprising first pixels;
obtaining a texture image of the first image, wherein the texture image comprises second pixels;
obtaining, in the texture image, a plurality of local texture images and respective weight values of the plurality of local texture images based on texture features of the second pixels, wherein textures in the plurality of local texture images have a same texture feature;
obtaining, based on the plurality of local texture images and the respective weight values, an image filter corresponding to the first image, wherein the image filter comprises respective filter parameters corresponding to each pixel of the first pixels, and wherein respective pixels of the first pixels that correspond to different texture features correspond to different filter parameters of the respective filter parameters; and
performing, based on the image filter, super-resolution processing on the first image to obtain a super-resolution image of the first image.

### Claim 2 (depends on 1)

The image processing method of claim 1, further comprising:
obtaining, in the texture image, C local texture images and respective weight values of the C local texture images based on texture features of the second pixels, wherein textures in the C local texture images have a same texture feature, wherein C=f2×t, wherein f is a preset amplification multiple, wherein t is a quantity of the respective filter parameters, wherein f is greater than 1, and wherein t is an integer greater than or equal to 1; and
a quantity of channels of the image filter is equal to C.

### Claim 3 (depends on 2)

The image processing method of claim 2, wherein an (x, y)th filter parameter in an ith channel of the image filter is a product of a pixel value of an (x, y)th pixel in an ith local texture image and a weight value of the ith local texture image, wherein i is a positive integer less than or equal to C, wherein x is a positive integer less than or equal to M, wherein y is a positive integer less than or equal to N, wherein M is a quantity of pixels horizontally comprised in the first image, wherein N is a quantity of pixels vertically comprised in the first image, and wherein M and N are integers greater than 1.

### Claim 4 (depends on 2)

The image processing method of claim 2, further comprising:
performing, based on a preset size, a compression processing on the first image to obtain a compressed first image of the preset size; and
obtaining the texture image based on the compressed first image.

### Claim 5 (depends on 1)

The image processing method of claim 1, wherein obtaining the image filter further comprises:
obtaining a recognition model by training a plurality of groups of samples, wherein each group of the groups of samples comprises a first sample image and a second sample image, wherein the first sample image and the second sample image have a same image content, and wherein a first resolution of the first sample image is higher than a second resolution of the second sample image; and
processing, using the recognition model, the first image to obtain the image filter.

### Claim 6 (depends on 5)

The image processing method of claim 5, further comprising:
performing, based on a preset size, compression processing on the first image to obtain a compressed first image of the preset size; and
processing the compressed first image using the recognition model.

### Claim 7 (depends on 1)

The image processing method of claim 1, further comprising:
obtaining a gradient image of the first image, wherein a size of the gradient image is the same as a size of the first image, wherein the size of the gradient image is M*N, and wherein M and N are both integers greater than 1;
processing, using the image filter, the gradient image to obtain a second image, wherein a size of the second image is (f*M)*(f*N), and wherein f is a preset amplification multiple;
amplifying the first image by f times to obtain a third image, wherein a size of the third image is (f*M)*(f*N); and
obtaining, based on the second image and the third image, the super-resolution image, wherein a size of the super-resolution image is (f*M)*(f*N).

### Claim 8 (depends on 7)

The image processing method of claim 7, further comprising:
further processing, using the image filter, the gradient image to obtain f2 sub-images, wherein a size of each of the f2 sub-images is (f*M)*(f*N); and
obtaining the second image based on the f2 sub-images.

### Claim 9 (depends on 8)

The image processing method of claim 8, further comprising further processing, using parameters in a (k*t+1)th channel to a (k+1)*tth channel in the image filter, the gradient image to obtain a kth sub-image, wherein k is sequentially set to 0, 1, . . . , and f2−1.

### Claim 10 (depends on 8)

The image processing method of claim 8, further comprising:
separately performing splicing processing on pixels that are at a same pixel position in the f2 sub-images to obtain M*N image blocks, wherein a size of each of the M*N image block is f*f; and
performing splice processing on the M*N image blocks based on pixel locations of pixels that are in each of the M*N image blocks in each of the f2 sub-images to obtain the second image.

### Claim 11

A computer system comprising:
a memory configured to store a computer program; and
one or more processors coupled to the memory, wherein when executed by the one or more processors, the computer program causes the computer system to:
obtain a first image that comprises first pixels;
obtain a texture image of the first image, wherein the texture image comprises second pixels;
obtain, in the texture image, a plurality of local texture images and respective weight values of the plurality of local texture images based on texture features of the second pixels, wherein textures in the plurality of local texture images have a same texture feature;
obtain, based on the plurality of local texture images and the respective weight values, an image filter, wherein the image filter comprises respective filter parameters corresponding to each pixel of the first pixels, and wherein respective pixels of the first pixels that correspond to different texture features correspond to different filter parameters of the respective filter parameters; and
perform, based on the image filter, super-resolution processing on the first image to obtain a super-resolution image of the first image.

### Claim 12 (depends on 11)

The computer system of claim 11, wherein when executed by the processor, the computer program further causes the computer system to:
obtain, in the texture image, C local texture images and respective weight values of the C local texture images based on texture features of the second pixels, wherein textures in the C local texture images have a same texture feature, wherein C=f2×t, wherein f is a preset amplification multiple, wherein t is a quantity of the respective filter parameter, wherein f is greater than 1, and wherein t is an integer greater than or equal to 1; and
a quantity of channels of the image filter is equal to C.

### Claim 13 (depends on 12)

The computer system of claim 12, wherein an (x, y)th filter parameter in an ith channel of the image filter is a product of a pixel value of an (x, y)th pixel in an ith local texture image and a weight value of the ith local texture image, wherein i is a positive integer less than or equal to C, wherein x is a positive integer less than or equal to M, wherein y is a positive integer less than or equal to N, wherein M is a quantity of pixels horizontally comprised in the first image, wherein N is a quantity of pixels vertically comprised in the first image, and wherein M and N are integers greater than 1.

### Claim 14 (depends on 12)

The computer system of claim 12, wherein when executed by the one or more processors, the computer program further causes the computer system to:
perform, based on a preset size, a compression processing on the first image to obtain a compressed first image of the preset size; and
obtain, based on the compressed first image, the texture image.

### Claim 15 (depends on 11)

The computer system of claim 11, wherein when executed by the one or more processors, the computer program further causes the computer system to:
obtain a recognition model by training a plurality of groups of samples, wherein each group of the groups of samples comprises a first sample image and a second sample image, wherein the first sample image and the second sample image have a same image content, and wherein a first resolution of the first sample image is higher than a second resolution of the second sample image; and
process, using the recognition model, the first image to obtain the image filter.

### Claim 16

A computer program product comprising computer-executable instructions that are stored on a non-transitory computer-readable storage medium and that, when executed by one or more processors, cause an apparatus to:
obtain a first image that comprises first pixels;
obtain a texture image of the first image, wherein the texture image comprises second pixels;
obtain, in the texture image, a plurality of local texture images and respective weight values of the plurality of local texture images based on texture features of the second pixels, wherein textures in the plurality of local texture images have a same texture feature;
obtain, based on the plurality of local texture images and the respective weight values, an image filter, wherein the image filter comprises respective filter parameters corresponding to each pixel of the first pixels, and wherein respective pixels of the first pixels that correspond to different texture features correspond to different filter parameters of the respective filter parameters; and
perform, based on the image filter, super-resolution processing on the first image to obtain a super-resolution image of the first image.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein when executed by the one or more processors, the computer-executable instructions further cause the apparatus to:
obtain, in the texture image, C local texture images and respective weight values of the C local texture images based on texture features of the second pixels, wherein textures in the C local texture images have a same texture feature, wherein C=f2×t, wherein f is a preset amplification multiple, wherein t is a quantity of the respective filter parameter, wherein f is greater than 1, and wherein t is an integer greater than or equal to 1; and
a quantity of channels of the image filter is equal to C.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein an (x, y)th filter parameter in an ith channel of the image filter is a product of a pixel value of an (x, y)th pixel in an ith local texture image and a weight value of the ith local texture image, wherein i is a positive integer less than or equal to C, wherein x is a positive integer less than or equal to M, wherein y is a positive integer less than or equal to N, wherein M is a quantity of pixels horizontally comprised in the first image, wherein N is a quantity of pixels vertically comprised in the first image, and wherein M and N are integers greater than 1.

### Claim 19 (depends on 17)

The computer program product of claim 17, wherein when executed by the one or more processors, the computer-executable instructions further cause the apparatus to:
perform, based on a preset size, compression processing on the first image to obtain a compressed first image of the preset size; and
obtain the texture image based on the compressed first image.

### Claim 20 (depends on 16)

The computer program product of claim 16, wherein when executed by the one or more processors, the computer-executable instructions further cause the apparatus to:
obtain a recognition model by training a plurality of groups of samples, wherein each group of the groups of samples comprises a first sample image and a second sample image, wherein the first sample image and the second sample image have a same image content, and wherein a first resolution of the first sample image is higher than a second resolution of the second sample image; and
process, using the recognition model, the first image to obtain the image filter.