# US 2025/0156995 A1 · Reference picture resampling (rpr) based super-resolution guided by partition information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156995 A1
- Document ID: US20250156995A1
- Status: Published application
- Kind: A1
- Application number: 19/003,957
- Filed: 2024-12-27
- Published: 2025-05-15
- Earliest claimed: 2022-07-06
- Assignees: Guangdong Oppo Mobile Telecommunications Corp., Ltd.
- Applicants: Guangdong Oppo Mobile Telecommunications Corp., Ltd.
- Inventors: Cheolkon JUNG; Qihui HAN
- Web page: https://patentgrep.com/patent/US20250156995A1

## CPC

- G06T 11/60: Creating or editing images; Combining images with text
- G06T 3/4046: using neural networks
- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 9/002: {using neural networks}
- G06V 10/40: Extraction of image or video features
- H04N 19/33: in the spatial domain
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/82: involving filtering within a prediction loop
- H04N 19/865: {with detection of the former encoding block subdivision in decompressed video}
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for video processing applied to a decoder includes (i) receiving an input image; (ii) processing the input image by one or more convolution layers; (iii) processing the input image by multiple residual blocks by using partition information of the input image as reference so as to obtain reference information features; (iv) generating different-scales features based on the reference information features; (v) processing the different-scales features by multiple convolutional layer sets; (vi) processing the different-scales features by reference spatial attention blocks (RSABs) so as to form a combined feature; and (vii) concatenating the combined feature with the reference information features so as to form an output image.

## Claims

### Claim 1

A method for video processing applied to a decoder, the method comprising:
receiving an input image;
processing the input image by one or more convolution layers;
processing the input image by multiple residual blocks by using partition information of the input image as reference so as to obtain reference information features;
generating different-scales features based on the reference information features;
processing the different-scales features by multiple convolutional layer sets;
processing the different-scales features by reference spatial attention blocks (RSABs) so as to form a combined feature; and
concatenating the combined feature with the reference information features so as to form an output image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more convolution layers belong to a feature extraction part of a framework.

### Claim 3 (depends on 1)

The method of claim 1, wherein the multiple residual blocks belong to a reference information generation (RIG) part of a framework.

### Claim 4 (depends on 3)

The method of claim 3, wherein the multiple residual blocks include eight residual blocks, and wherein the first four residual blocks are used for predicting coding-tree-unit (CTU) partition information from the one or more convolution layers.

### Claim 5 (depends on 4)

The method of claim 4, wherein the RIG part further includes the multiple convolutional layer sets, and wherein each of the multiple convolutional layer sets includes a convolutional layer with stride 2 and a convolutional layer followed by a rectified linear unit (ReLU).

### Claim 6 (depends on 1)

The method of claim 1, further comprising processing the different-scales features by dilated convolutional layers based dense blocks with channel attention (DDBCAs) so as to form the combined feature.

### Claim 7 (depends on 6)

The method of claim 6, wherein the DDBCAs and the RSABs belong to a mutual information processing (MIP) part of a framework.

### Claim 8 (depends on 7)

The method of claim 7, wherein the MIP part includes four scales configured to generating the different-scales features.

### Claim 9 (depends on 8)

The method of claim 8, wherein at least one of the four scales includes two DDBCAs followed by one RSAB.

### Claim 10 (depends on 8)

The method of claim 8, wherein one of the four scales includes four DDBCAs followed by one RSAB.

### Claim 11 (depends on 1)

The method of claim 1, wherein the combined feature is concatenated by a reconstruction part of a framework.

### Claim 12 (depends on 11)

The method of claim 11, wherein the reconstruction part includes three branch paths for processing luma and chroma components, respectively.

### Claim 13

A system for video processing, the system comprising:
a processor; and
a memory configured to store instructions, when executed by the processor, to:
receive an input image;
process the input image by one or more convolution layers;
process the input image by multiple residual blocks by using partition information of the input image as reference so as to obtain reference information features;
generate different-scales features based on the reference information features;
process the different-scales features by multiple convolutional layer sets;
process the different-scales features by reference spatial attention blocks (RSABs) so as to form a combined feature; and
concatenate the combined feature with the reference information features so as to form an output image.

### Claim 14 (depends on 13)

The system of claim 13, wherein the one or more convolution layers belong to a feature extraction part of a framework.

### Claim 15 (depends on 13)

The system of claim 13, wherein the multiple residual blocks belong to a reference information generation (RIG) part of a framework.

### Claim 16 (depends on 15)

The system of claim 15, wherein the multiple residual blocks include eight residual blocks, wherein the first four residual blocks are used for predicting coding-tree-unit (CTU) partition information from the one or more convolution layers, wherein the RIG part further includes the multiple convolutional layer sets, and wherein each of the multiple convolutional layer sets includes a convolutional layer with stride 2 and a convolutional layer followed by a rectified linear unit (ReLU).

### Claim 17 (depends on 13)

The system of claim 13, wherein the different-scales features is processed by dilated convolutional layers based dense blocks with channel attention (DDBCAs) so as to form the combined feature.

### Claim 18 (depends on 17)

The system of claim 17, wherein the DDBCAs and the RSABs belong to a mutual information processing (MIP) part of a framework.

### Claim 19 (depends on 17)

The system of claim 17, wherein the MIP part includes four scales configured to generating the different-scales features.

### Claim 20

A method for video processing applied to an encoder, the method comprising:
receiving an input image;
processing the input image by one or more convolution layers;
processing the input image by multiple residual blocks by using partition information of the input image as reference so as to obtain reference information features;
generating different-scales features based on the reference information features;
processing the different-scales features by multiple convolutional layer sets;
processing the different-scales features by reference spatial attention blocks (RSABs) and dilated convolutional layers based dense blocks with channel attention (DDBCAs) so as to form a combined feature; and
concatenating the combined feature with the reference information features so as to form an output image.
