# US 2023/0067568 A1 · Frame Sequence Quality Booster under Uneven Quality Conditions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0067568 A1
- Document ID: US20230067568A1
- Status: Published application
- Kind: A1
- Application number: 17/894,947
- Filed: 2022-08-24
- Published: 2023-03-02
- Earliest claimed: 2021-08-26
- Assignees: Mediatek Inc.
- Applicants: MediaTek Inc.
- Inventors: Yao-Sheng Wang; Pei-Kuei Tsung; Chiung-Fu Chen; Wai Mun Wong; Chao-Min Chang; Yu-Sheng Lin; Chiani Lu; Chih-Cheng Chen
- Web page: https://patentgrep.com/patent/US20230067568A1

## CPC

- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30168: Image quality inspection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The quality of a frame sequence is enhanced by a booster engine collaborating with a first stage circuit. The first stage circuit adjusts the quality degradation of the frame sequence when a condition in constrained resources is detected. The quality degradation includes at least one of uneven resolution and uneven frame per second (FPS). The booster engine receives the frame sequence from the first stage circuit, and generates an enhanced frame sequence based on the frame sequence for transmission to a second stage circuit.

## Claims

### Claim 1

A method for enhancing quality of a frame sequence, comprising:
adjusting, by a first stage circuit, quality degradation of the frame sequence when a condition in constrained resources is detected, the quality degradation including at least one of uneven resolution and uneven frame per second (FPS);
receiving, by a booster engine, the frame sequence from the first stage circuit; and
generating, by the booster engine, an enhanced frame sequence based on the frame sequence for transmission to a second stage circuit.

### Claim 2 (depends on 1)

The method of claim 1, wherein the constrained resources include one or more of computational resources, power resources, and transmission bandwidth.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the enhanced frame sequence further comprises:
calculating an optical flow for a current frame using a frame preceding the current frame as a reference frame.

### Claim 4 (depends on 1)

The method of claim 1, wherein generating the enhanced frame sequence further comprises:
calculating an optical flow for a current frame using a previous frame having better quality than a quality threshold as a reference frame.

### Claim 5 (depends on 1)

The method of claim 1, wherein adjusting the quality degradation further comprises:
dynamically changing frame quality during frame sequence generation in response to resource constraints.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
activating the booster engine by sending a help request from the first stage circuit to the booster engine, the help request indicating the quality degradation in the plurality of frames; and
performing, by the booster engine in response to the help request, image alignment between a current frame and a reference frame, and frame quality optimization including at least super-resolution and inpainting.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
detecting, by the booster engine, the quality degradation in a current frame; and
performing, by the booster engine, image alignment between the current frame and a reference frame, and frame quality optimization including at least super-resolution and inpainting.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
optimizing, by the booster engine, quality of a received frame in the frame sequence;
comparing, by the booster engine, the optimized quality with a threshold; and
reverting back to the quality of the received frame when the optimized quality is below the threshold.

### Claim 9 (depends on 1)

The method of claim 1, wherein the first stage circuit, the booster engine, and the second stage circuit are located within a same electronic device.

### Claim 10 (depends on 1)

The method of claim 1, wherein the first stage circuit and the second stage circuit are in two electronic devices coupled to each other by a transmission network.

### Claim 11

A system operative to enhance quality of a frame sequence, comprising:
a first stage circuit to adjust quality degradation of the frame sequence when a condition in constrained resources is detected, the quality degradation including at least one of uneven resolution and uneven frame per second (FPS); and
a booster engine to receive the frame sequence from the first stage circuit, and to generate an enhanced frame sequence based on the frame sequence for transmission to a second stage circuit.

### Claim 12 (depends on 11)

The system of claim 11, wherein the constrained resources include one or more of computational resources, power resources, and transmission bandwidth.

### Claim 13 (depends on 11)

The system of claim 11, wherein the booster engine is further operative to calculate an optical flow for a current frame using a frame preceding the current frame as a reference frame.

### Claim 14 (depends on 11)

The system of claim 11, wherein the booster engine is further operative to calculate an optical flow for a current frame using a previous frame having better quality than a quality threshold as a reference frame.

### Claim 15 (depends on 11)

The system of claim 11, wherein the first stage circuit is further operative to dynamically change frame quality during frame sequence generation in response to resource constraints.

### Claim 16 (depends on 11)

The system of claim 11, wherein the first stage circuit is further operative to activate the booster engine by sending a help request that indicates the quality degradation in the plurality of frames, and the booster engine in response to the help request is operative to perform image alignment between a current frame and a reference frame, and frame quality optimization including at least super-resolution and inpainting.

### Claim 17 (depends on 11)

The system of claim 11, wherein the booster engine is further operative to detect the quality degradation in a current frame, and perform image alignment between the current frame and a reference frame, and frame quality optimization including at least super-resolution and inpainting.

### Claim 18 (depends on 11)

The system of claim 11, wherein the booster engine is further operative to:
optimize quality of a received frame in the frame sequence;
compare the optimized quality with a threshold; and
revert back to the quality of the received frame when the optimized quality is below the threshold.

### Claim 19 (depends on 11)

The system of claim 11, wherein the first stage circuit, the booster engine, and the second stage circuit are located within a same electronic device.

### Claim 20 (depends on 11)

The system of claim 11, wherein the first stage circuit and the second stage circuit are in two electronic devices coupled to each other by a transmission network.
