# US 2024/0005458 A1 · Region-of-interest (roi)-based image enhancement using a residual network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005458 A1
- Document ID: US20240005458A1
- Status: Published application
- Kind: A1
- Application number: 17/855,288
- Filed: 2022-06-30
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: ATI Technologies ULC
- Applicants: Ati Technologies Ulc
- Inventors: TUNG CHUEN KWONG; YING LIU
- Web page: https://patentgrep.com/patent/US20240005458A1

## CPC

- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Region-of-interest (ROI)-based image enhancement using a residual network, including: generating, based on an input image and a residual path of a residual network, a first output corresponding to a region-of-interest of the input image; generating, based on the input image and a skip path of the residual network, a second output; and generating an output image based on the first output and the second output.

## Claims

### Claim 1

A method of region-of-interest (ROI)-based image enhancement using a residual network, the method comprising:
generating, based on an input image and a residual path of a residual network, a first output corresponding to a region-of-interest of the input image;
generating, based on the input image and a skip path of the residual network, a second output; and
generating an output image based on the first output and the second output.

### Claim 2 (depends on 1)

The method of claim 1, wherein the region-of-interest comprises a segmentation mask.

### Claim 3 (depends on 1)

The method of claim 1, wherein the residual network comprises an image enhancement neural network.

### Claim 4 (depends on 3)

The method of claim 3, further comprising performing a backpropagation of the residual path based on the region-of-interest, including identifying, for each layer of the residual path, a corresponding input feature map subspace based on the region-of-interest.

### Claim 5 (depends on 4)

The method of claim 4, wherein performing the backpropagation further comprises storing, for each layer of the residual path, subspace metadata describing the corresponding input feature map subspace.

### Claim 6 (depends on 5)

The method of claim 5, wherein generating the first output comprises processing, for each layer of the residual path, the corresponding input feature map subspace described in the subspace metadata.

### Claim 7 (depends on 1)

The method of claim 1, wherein the input image comprises a frame of video data, and wherein the region-of-interest comprises a dynamic region-of-interest that is variable across a plurality of frames of the video data.

### Claim 8

An apparatus for region-of-interest (ROI)-based image enhancement using a residual network, the apparatus comprising:
a computer processor;
a computer memory operatively coupled to the computer processor; and
an image enhancement circuit, wherein the image enhancement circuit comprises logic configured to:
generate, based on an input image and a residual path of a residual network, a first output corresponding to a region-of-interest of the input image;
generate, based on the input image and a skip path of the residual network, a second output; and
generate an output image based on the first output and the second output.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the region-of-interest comprises a segmentation mask.

### Claim 10 (depends on 8)

The apparatus of claim 8, wherein the residual network comprises an image enhancement neural network.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein image enhancement circuit is further configured to perform a backpropagation of the residual path based on the region-of-interest, including identifying, for each layer of the residual path, a corresponding input feature map subspace based on the region-of-interest.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein performing the backpropagation further comprises storing, for each layer of the residual path, subspace metadata describing the corresponding input feature map subspace.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein generating the first output comprises processing, for each layer of the residual path, the corresponding input feature map subspace described in the subspace metadata.

### Claim 14 (depends on 8)

The apparatus of claim 8, wherein the input image comprises a frame of video data, and wherein the region-of-interest comprises a dynamic region-of-interest that is variable across a plurality of frames of the video data.

### Claim 15

A computer program product disposed upon a non-transitory computer readable medium, the computer program product comprising computer program instructions for region-of-interest (ROI)-based image enhancement using a residual network that, when executed, cause a computer system to perform steps comprising:
generating, based on an input image and a residual path of a residual network, a first output corresponding to a region-of-interest of the input image;
generating, based on the input image and a skip path of the residual network, a second output; and
generating an output image based on the first output and the second output.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the region-of-interest comprises a segmentation mask.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein the residual network comprises an image enhancement neural network.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein the steps further comprise performing a backpropagation of the residual path based on the region-of-interest, including identifying, for each layer of the residual path, a corresponding input feature map subspace based on the region-of-interest.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein performing the backpropagation further comprises storing, for each layer of the residual path, subspace metadata describing the corresponding input feature map subspace.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein generating the first output comprises processing, for each layer of the residual path, the corresponding input feature map subspace described in the subspace metadata.
