# US 2025/0061686 A1 · An image processing method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0061686 A1
- Document ID: US20250061686A1
- Status: Published application
- Kind: A1
- Application number: 18/725,661
- Filed: 2022-12-27
- Published: 2025-02-20
- Earliest claimed: 2021-12-28
- Assignees: Beijing Zitiao Network Technology Co., Ltd.
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Hang DONG
- Web page: https://patentgrep.com/patent/US20250061686A1

## CPC

- G06V 10/806: {of extracted features}
- G06F 18/00: Pattern recognition
- G06F 18/253: {of extracted features}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06V 10/52: Scale-space analysis, e.g. wavelet analysis (multi-scale boundary representations G06V10/42)
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments of the present disclosure relate to the technical field of image processing, and provide an image processing method and apparatus. The method includes: respectively performing feature extraction on an image to be processed from a plurality of different spatial scales to obtain a target feature and at least one feature to be fused; fusing the target feature and the at least one feature to be fused to obtain a first feature; extracting a high-frequency feature and a low-frequency feature from the target feature; processing the high-frequency feature on the basis of a residual dense block (RDB) to obtain a second feature; fusing the low-frequency feature and the at least one feature to be fused to obtain a third feature; combining the first feature, the second feature and the third feature to obtain a fused feature; and processing the image to be processed on the basis of the fused feature.

## Claims

### Claim 1

An image processing method, comprising:
performing feature extraction on an image to be processed from a plurality of different spatial scales respectively, to obtain a target feature and at least one feature to be fused;
fusing the target feature and the at least one feature to be fused to obtain a first feature;
extracting high-frequency features and low-frequency features from the target feature;
processing the high-frequency features based on a residual dense block (RDB) to obtain a second feature;
fusing the low-frequency features and the at least one feature to be fused to obtain a third feature;
combining the first feature, the second feature and the third feature to obtain a fused feature; and
processing the image to be processed based on the fused feature.

### Claim 2 (depends on 1)

The method according to claim 1, the extracting high-frequency features and low-frequency features from the target feature includes:
performing discrete wavelet decomposition on the target feature to obtain a fourth feature;
determining features of first preset number of channels of the fourth feature as the low-frequency features, and features of other channels in the fourth feature except the low-frequency features as the high-frequency features.

### Claim 3 (depends on 2)

The method according to claim 2, after extracting high-frequency features and low-frequency features in the target feature, the method further includes:
processing the high-frequency features and the low-frequency features respectively through a convolution layer to reduce the number of channels of the high-frequency features and the low-frequency features to a preset value.

### Claim 4 (depends on 1)

The method according to claim 1, the fusing the low-frequency features and the at least one feature to be fused to obtain a third feature includes:
sorting the at least one feature to be fused in descending order according to the spatial scale difference between the at least one feature to be fused and the low-frequency features, to obtain a first sorting result;
fusing the first feature to be fused and the low-frequency feature to obtain the fused feature corresponding to the first feature to be fused, the first feature to be fused being the first feature to be fused in the first sorting result;
fusing other features to be fused in the first sorting result and the fused feature corresponding to the previous feature to be fused one by one, to obtain fused features corresponding to other features to be fused in the first sorting result; and
determining the corresponding fused feature of the last feature to be fused in the first sorting result as the third feature.

### Claim 5 (depends on 4)

The method according to claim 4, the fusing the first feature to be fused and the low-frequency feature to obtain the fused feature corresponding to the first feature to be fused includes:
sampling the low-frequency feature as a first sampled feature; the first sampled feature having the same spatial scale as the first feature to be fused;
calculating the difference between the first sampled feature and the first feature to be fused, to obtain a first difference feature;
sampling the first difference feature as a second sampled feature; the second sampled feature having the same spatial scale as the low-frequency feature; and
additively fusing the low-frequency feature and the second sampled feature to generate a fused feature corresponding to the first feature to be fused.

### Claim 6 (depends on 4)

The method according to claim 4, the fusing other features to be fused in the first sorting result and the fused feature corresponding to the previous feature to be fused one by one, to obtain fused features corresponding to other features to be fused in the first sorting result includes:
sampling the fused feature corresponding to the m−1-th feature to be fused in the first sorting result as a third sampled feature; the third sampled feature having the same spatial scale as the m-th feature to be fused in the first sorting result, m being an integer greater than 1;
calculating the difference between the m-th feature to be fused and the third sampled feature to obtain a second difference feature;
sampling the second difference feature as a fourth sampled feature; the fourth sampled feature having the same spatial scale as the fused feature corresponding to the m−1-th feature to be fused; and
additively fusing the fused feature corresponding to the m−1-th feature to be fused and the fourth sampled feature to generate a fused feature corresponding to the m-th feature to be fused.

### Claim 7 (depends on 1)

The method according to claim 1, the fusing the target feature and the at least one feature to be fused to obtain a first feature includes:
dividing the target feature into a fifth feature and a sixth feature;
processing the fifth feature based on a residual dense block (RDB) to obtain a seventh feature;
fusing the sixth feature and the at least one feature to be fused to obtain an eighth feature;
combining the seventh feature and the eighth feature to generate the first feature.

### Claim 8 (depends on 7)

The method according to claim 7, fusing the sixth feature and the at least one feature to be fused to obtain an eighth feature includes:
sorting the at least one feature to be fused in descending order according to the spatial scale difference between the at least one feature to be fused and the sixth feature, to obtain a second sorting result;
fusing a second feature to be fused and the sixth feature to obtain the fused feature corresponding to the second feature to be fused, the second feature to be fused being the first feature to be fused in the second sorting result;
fusing other features to be fused in the second sorting result and the fused feature corresponding to the previous feature to be fused one by one, to obtain fused features corresponding to the other features to be fused in the second sorting result; and
determining the fused feature corresponding to the last feature to be fused in the second sorting result as the eighth feature.

### Claim 9 (depends on 8)

The method according to claim 8, the fusing a second feature to be fused and the sixth feature to obtain the fused feature corresponding to the second feature to be fused includes:
sampling the sixth feature as a fifth sampled feature, the fifth sampled feature having the same spatial scale as the second feature to be fused;
calculating the difference between the fifth sampled feature and the first feature to be fused in the second sorting result, to obtain the third difference feature;
sampling the third difference feature as a sixth sampled feature, the sixth sampled feature having the same spatial scale as the sixth feature; and
additively fusing the sixth feature and the sixth sampled feature to generate a fused feature corresponding to the second feature to be fused.

### Claim 10 (depends on 8)

The method according to claim 8, the fusing other features to be fused in the second sorting result and the fused feature corresponding to the previous feature to be fused one by one, to obtain fused features corresponding to other features to be fused in the second sorting result includes:
sampling the fused feature corresponding to the n−1-th feature to be fused in the second sorting result as a seventh sampled feature; the seventh sampled feature having the same spatial scale as the n-th feature to be fused in the second sorting result, n being an integer greater than 1;
calculating the difference between the n-th feature to be fused and the seventh sampled feature to obtain a fourth difference feature;
sampling the fourth difference feature as an eighth sampled feature, the eighth sampled feature having the same spatial scale as the fused feature corresponding to the n−1-th feature to be fused; and
additively fusing the fused feature corresponding to the n−1-th feature to be fused and the eighth sampled feature to generate a fused feature corresponding to the n-th feature to be fused.

### Claim 11 (depends on 7)

The method according to claim 7, the dividing the target feature into a fifth feature and a sixth feature includes:
dividing the target feature into a fifth feature and a sixth feature based on feature channels of the target feature.

### Claim 12 (depends on 1)

The method according to claim 1, comprising:
processing an image to be processed through an encoding module to obtain an encoded feature; wherein the encoding module includes L cascaded encoders with different spatial scales, and the i-th encoder is used to perform feature extraction on the image to be processed to obtain an image feature on the i-th encoder, and obtain fused features output by all encoders before the i-th encoder, and obtain the fused feature of the i-th encoder, and output the fused features of the i-th encoder to all encoders after the i-th encoder, L and i both being positive integers, and i≤L;
processing the encoded feature through a feature restoration module composed of at least one residual block (RDB) to obtain a restored feature;
processing the restored feature through a decoding module to obtain a processing result image of the image to be processed; wherein the decoding module includes L cascaded decoders with different spatial scales, and the j-th decoder is used to fuse an image feature of the encoding module on the j-th encoder and the fusion results output by all decoders before the j-th decoder, generate a fusion result of the j-th decoder, and output the fusion result of the j-th decoder to all decoders after the j-th decoder.

### Claim 13 (depends on 12)

The method according to claim 12, the processing the restored feature through a decoding module to obtain a processing result image of the image to be processed includes:
dividing the image feature on the j-th decoder into a ninth feature and a tenth feature;
processing the ninth feature based on a residual dense block (RDB) to obtain an eleventh feature;
fusing the tenth feature and fusion results output by all decoders before the j-th decoder to obtain a twelfth feature;
combining the eleventh feature and the twelfth feature to generate a fusion result of the j-th decoder.

### Claim at position 14

14-15. (canceled)

### Claim 16

An electronic device, comprising: a memory and a processor, wherein the memory is configured to store a computer program; the processor is configured to, when calling the computer program, cause the electronic device to implement an image processing method, comprising:
performing feature extraction on an image to be processed from a plurality of different spatial scales respectively, to obtain a target feature and at least one feature to be fused;
fusing the target feature and the at least one feature to be fused to obtain a first feature;
extracting high-frequency features and low-frequency features from the target feature;
processing the high-frequency features based on a residual dense block (RDB) to obtain a second feature;
fusing the low-frequency features and the at least one feature to be fused to obtain a third feature;
combining the first feature, the second feature and the third feature to obtain a fused feature; and
processing the image to be processed based on the fused feature.

### Claim at position 16

17-18. (canceled)

### Claim 19 (depends on 16)

The device according to claim 16, wherein the extracting high-frequency features and low-frequency features from the target feature includes:
performing discrete wavelet decomposition on the target feature to obtain a fourth feature;
determining features of first preset number of channels of the fourth feature as the low-frequency features, and features of other channels in the fourth feature except the low-frequency features as the high-frequency features.

### Claim 20 (depends on 19)

The device according to claim 19, wherein after extracting high-frequency features and low-frequency features in the target feature, the method further includes:
processing the high-frequency features and the low-frequency features respectively through a convolution layer to reduce the number of channels of the high-frequency features and the low-frequency features to a preset value.

### Claim 21 (depends on 16)

The device according to claim 16, wherein the fusing the low-frequency features and the at least one feature to be fused to obtain a third feature includes:
sorting the at least one feature to be fused in descending order according to the spatial scale difference between the at least one feature to be fused and the low-frequency features, to obtain a first sorting result;
fusing the first feature to be fused and the low-frequency feature to obtain the fused feature corresponding to the first feature to be fused, the first feature to be fused being the first feature to be fused in the first sorting result;
fusing other features to be fused in the first sorting result and the fused feature corresponding to the previous feature to be fused one by one, to obtain fused features corresponding to other features to be fused in the first sorting result; and
determining the corresponding fused feature of the last feature to be fused in the first sorting result as the third feature.

### Claim 22 (depends on 21)

The device according to claim 21, wherein the fusing the first feature to be fused and the low-frequency feature to obtain the fused feature corresponding to the first feature to be fused includes:
sampling the low-frequency feature as a first sampled feature; the first sampled feature having the same spatial scale as the first feature to be fused;
calculating the difference between the first sampled feature and the first feature to be fused, to obtain a first difference feature;
sampling the first difference feature as a second sampled feature; the second sampled feature having the same spatial scale as the low-frequency feature; and
additively fusing the low-frequency feature and the second sampled feature to generate a fused feature corresponding to the first feature to be fused.

### Claim 23

A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a computing device, causes the computing device to implement an image processing method, the image processing method comprises:
performing feature extraction on an image to be processed from a plurality of different spatial scales respectively, to obtain a target feature and at least one feature to be fused;
fusing the target feature and the at least one feature to be fused to obtain a first feature;
extracting high-frequency features and low-frequency features from the target feature;
processing the high-frequency features based on a residual dense block (RDB) to obtain a second feature;
fusing the low-frequency features and the at least one feature to be fused to obtain a third feature;
combining the first feature, the second feature and the third feature to obtain a fused feature; and
processing the image to be processed based on the fused feature.

### Claim 24 (depends on 23)

The medium according to claim 23, wherein the extracting high-frequency features and low-frequency features from the target feature includes:
performing discrete wavelet decomposition on the target feature to obtain a fourth feature;
determining features of first preset number of channels of the fourth feature as the low-frequency features, and features of other channels in the fourth feature except the low-frequency features as the high-frequency features.
