# US 2025/0097440 A1 · Learned image compression and decompression using long and short attention module

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0097440 A1
- Document ID: US20250097440A1
- Status: Published application
- Kind: A1
- Application number: 18/956,949
- Filed: 2024-11-22
- Published: 2025-03-20
- Earliest claimed: 2022-05-23
- Assignees: Guangdong Oppo Mobile Telecommunications Corp., Ltd.
- Applicants: Guangdong Oppo Mobile Telecommunications Corp., Ltd.
- Inventors: Cheolkon JUNG; Zenghui DUAN
- Web page: https://patentgrep.com/patent/US20250097440A1

## CPC

- G06V 10/82: using neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/08: Learning methods
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/806: {of extracted features}
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence)
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for feature extraction using a neural network includes: extracting a first set of features from an input set of features by processing said input set of features through at least two residual blocks of the neural network connected successively to each other; extracting a second set of features from the input set of features, said extracting the second set of features comprising the steps of: extracting a third set of features and a fourth set of features from the input set of features by a non-local attention processing, implementing a group convolution by a multi-head attention mechanism to the input set of features, the third set of features and the fourth set of features to obtain the second set of features.

## Claims

### Claim 1

A method for feature extraction using a neural network, the method comprising:
extracting a first set of features from an input set of features by processing said input set of features through at least two residual blocks of the neural network connected successively to each other;
extracting a second set of features from the input set of features, said extracting the second set of features comprising the steps of:
extracting a third set of features and a fourth set of features from the input set of features by a non-local attention processing,
implementing a group convolution by a multi-head attention mechanism to the input set of features, the third set of features and the fourth set of features to obtain the second set of features.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the input set of features is a multidimensional vector representing an image data.

### Claim 3 (depends on 2)

The method of claim 2, wherein
the first set of features is a set of local features of the image data and the second set of features is a set of global features of the image data.

### Claim 4 (depends on 1)

The method according to claim 1, comprising further the steps of:
obtaining a fifth set of features by processing the second set of features by at least two convolutional layers of the neural network.

### Claim 5 (depends on 4)

The method according to claim 4, further comprising the steps of:
performing cascading processing on the first set of features and the fifth set of features, and
fusing the first set of features and fifth set of features after cascading processing to thereby obtain a mixed set of features.

### Claim 6 (depends on 5)

The method according to claim 5, further comprising:
performing convolution processing on the mixed set of features by at least one convolutional layer of the neural network to obtain a sixth set of features, and
outputting the sixth set of features.

### Claim 7 (depends on 1)

The method according to claim 1, wherein implementing the group convolution comprises the steps of:
multiplying the third set of features and the fourth set of features,
adding the input set of features to the multiplication of the third set of features and the fourth set of features to obtain a resultant set of features, and
concatenating the resultant set of features by the number of heads of the multi-head attention mechanism to obtain the second set of features.

### Claim 8 (depends on 1)

The method according to claim 1, wherein extracting the third set of features comprises:
processing the input set of features through at least three residual blocks of the neural network successively connected to each other.

### Claim 9 (depends on 1)

The method according to claim 1, wherein extracting the fourth set of features comprises:
processing the input set of features by at least three residual blocks of the neural network successively connected to each other;
processing the output of the last of the at least three residual blocks by a convolutional layer of the neural network; and
performing activation processing by an activation layer of the neural network to the output of the at least one convolutional layer.

### Claim 10 (depends on 1)

A method for learned image compression using a neural network, the method comprising performing the steps of:
extracting a set of features (x) from an input image data to be compressed; and
extracting a set of features (y) indicating a latent representation of the input image data to be compressed from the set of features (x) extracted from the input image data to be compressed by performing the steps of:
performing at least four steps of downsampling convolution processing by at least four convolutional layers of the neural network arranged in a stream like manner on the extracted set of features (x) from the input image data, and
performing the steps of the feature extraction method of claim 1 at least once after two steps of downsampling convolution processing by two convolutional layers from the at least four convolutional layer by using as the input set of features a set of features based on the output of the second convolutional layer from the two convolutional layers to thereby extract the set of features (y) indicating the latent representation of the input image data; and
the method further comprising the step of:
outputting the extracted set of features (y) indicating a latent representation of the input image data.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
inserting a step of processing by at least one residual block of the neural network at least between two steps of downsampling processing by two convolutional layers of the at least four downsampling convolutional layers arranged in a stream like manner.

### Claim 12 (depends on 1)

A method for learned image decompression using a neural network, the method comprising:
extracting a set of features from an input set of features to be decompressed; and
extracting a set of features indicating a reconstructed image of the input image data from the extracted set of features by performing the steps of:
performing the steps of the feature extraction method of claim 1 using as the input set of features the extracted set of features from the input set of features to be decompressed,
performing upsampling convolution processing by at least four convolutional layers of the neural network arranged in a stream like manner on the extracted set of features from the input set of features to be decompressed, and
performing the feature extraction method of claim 1 at least once after two steps of upsampling convolution processing by two convolutional layers from the at least four convolutional layer by using as the input set of features a set of features based on the output of the second convolutional layer from the two convolutional layers to thereby extract a set of features indicating a reconstructed image of the input image data; and
the method further comprising:
outputting the extracted set of features indicating a reconstructed image of the input image data.

### Claim 13 (depends on 12)

The method of claim 12, wherein the method further comprises:
inserting a step of processing by at least one residual block of the neural network at least once between two steps of upsampling processing by two convolutional layers of the at least four upsampling convolutional layers arranged in a stream like manner.

### Claim 14 (depends on 1)

The method according to claim 1, wherein performing the step of processing by a residual block comprises:
performing convolution by at least two convolutional layers of the neural network;
performing activation by at least one activation layer of the neural network after each of the at least two convolutional layers of the neural network;
implementing a skip connection providing an alternative path from an input of the residual block to an output of the residual block; and
obtaining the output of the residual block by fusing the skip connection and the last activation layer of the residual block.

### Claim 15 (depends on 1)

A non-transitory computer readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 1.

### Claim 16 (depends on 10)

A non-transitory computer readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 10.

### Claim 17 (depends on 12)

A non-transitory computer readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 12.
