# US 2025/0166207 A1 · Image processing apparatus and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166207 A1
- Document ID: US20250166207A1
- Status: Published application
- Kind: A1
- Application number: 18/635,021
- Filed: 2024-04-15
- Published: 2025-05-22
- Earliest claimed: 2023-11-16
- Assignees: Realtek Semiconductor Corp.
- Applicants: Realtek Semiconductor Corporation
- Inventors: Chung-Ping YU
- Web page: https://patentgrep.com/patent/US20250166207A1

## CPC

- G06T 7/223: using block-matching
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/248: {involving reference images or patches}
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/30204: Marker
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing apparatus and method are provided. An image processing device is configured to execute the following operations. The apparatus marks a first periodic block in a down-sized current frame with a first label. The apparatus performs a first motion estimation on the down-sized current frame and a down-sized reference frame based on the first label to generate first motion vectors. The apparatus marks an n-th periodic block having another periodic feature in a current frame with an n-th label. The apparatus performs an n-th motion estimation on the current frame and a reference frame based on the n-th label to generate n-th motion vectors. The apparatus performs a motion compensation on the current frame and the reference frame based on the n-th motion vectors to generate a compensated frame.

## Claims

### Claim 1

An image processing apparatus, comprising:
a storage, configured to store a current frame and a reference frame; and
a processor, coupled to the storage, configured to execute the following operations:
downsizing the current frame and the reference frame to generate a down-sized current frame and a down-sized reference frame respectively;
marking at least one of a plurality of first blocks in the down-sized current frame as at least one first periodic pattern block with at least one first label respectively, wherein the at least one first periodic pattern block has a first periodic feature;
performing a first motion estimation on the down-sized current frame and the down-sized reference frame based on the at least one first label to generate a plurality of first motion vectors;
marking at least one of a plurality of n-th blocks in the current frame as at least one n-th periodic pattern block with at least one n-th label respectively, wherein the at least one n-th periodic pattern block has a n-th periodic feature;
performing an n-th motion estimation on the current frame and the reference frame based on the first motion vectors and the at least one n-th label to generate a plurality of n-th motion vectors; and
performing a motion compensation on the current frame and the reference frame based on the n-th motion vectors to generate a compensated frame between the current frame and the reference frame.

### Claim 2 (depends on 1)

The image processing apparatus of claim 1, wherein the operation of marking the at least one first periodic pattern block further comprises:
comparing a plurality of pixels and a plurality of shifted pixels in the down-sized current frame to calculate a plurality of differences, wherein the shifted pixels are the pixels after being shifted based on a plurality of different shifting values, and each of the differences is corresponding to one of the different shifting values; and
determining whether to mark the pixels as the at least one first periodic pattern block or not based on a minimum difference of the differences and a threshold.

### Claim 3 (depends on 2)

The image processing apparatus of claim 2, wherein the operation of marking the at least one first periodic pattern block further comprises:
calculating an interleaving frequency of the pixels corresponding to a mean value of the pixels; and
determining whether to mark the pixels as the at least one first periodic pattern block or not based on the minimum difference, the threshold, the interleaving frequency, and a most similar shifting value corresponding to the minimum difference.

### Claim 4 (depends on 2)

The image processing apparatus of claim 2, wherein the first motion estimation comprises a plurality of scanning operations, and the scanning operations further comprises:
generating a searching window corresponding to each of the first blocks in the down-sized current frame;
adjusting the corresponding searching window based on the most similar shifting value and a shifting direction of each of the at least one first periodic pattern block;
generating a plurality of candidate vectors in the searching window; and
selecting the first motion vectors corresponding to each of the at least one first periodic pattern block from the candidate vectors.

### Claim 5 (depends on 1)

The image processing apparatus of claim 1, wherein the n-th motion estimation comprises a plurality of a plurality of scanning operations, and the scanning operations further comprises:
generating a searching window corresponding to each of the n-th blocks in the current frame;
generating a plurality of candidate vectors in the searching window; and
selecting the n-th motion vectors corresponding to each of the at least one n-th periodic pattern block from the candidate vectors.

### Claim 6 (depends on 5)

The image processing apparatus of claim 5, wherein the operation of selecting one of the n-th motion vectors further comprises:
in response to the candidate vectors corresponding to one of the at least one n-th periodic pattern block, calculating a difference between a reference motion vector corresponding to the one of the at least one n-th periodic pattern block and each of the candidate vectors;
calculating a punish value of each of the candidate vectors based on the difference corresponding to each of the candidate vectors, wherein the difference and the punish value are positively related;
reducing a weight of each of the candidate vectors based on the punish value of each of the candidate vectors; and
selecting one of the n-th motion vectors from the candidate vectors based on the weight of each of the candidate vectors.

### Claim 7 (depends on 6)

The image processing apparatus of claim 6, wherein the reference motion vector is one of a plurality of n−1-th motion vectors corresponding to the one of the at least one n-th periodic pattern block.

### Claim 8 (depends on 6)

The image processing apparatus of claim 6, wherein the reference motion vector is a regional motion vector corresponding to the one of the at least one n-th periodic pattern block, wherein the regional motion vector is corresponding to the one of the at least one n-th periodic pattern block and at least one periodic pattern block nearby the one of the at least one n-th periodic pattern block.

### Claim 9 (depends on 6)

The image processing apparatus of claim 6, wherein the operation of selecting one of the n-th motion vectors further comprises:
in response to the candidate vectors corresponding to one of the at least one n-th periodic pattern block, removing a spatial candidate vector, a temporal candidate vector, and a random candidate vector from the candidate vectors; and
selecting one of the n-th motion vectors from the remaining candidate vectors.

### Claim 10 (depends on 1)

The image processing apparatus of claim 1, wherein the operation of marking the at least one of the n-th blocks in the current frame as the at least one n-th periodic pattern block with the at least one n-th label respectively further comprises:
performing an OR operation on the at least one n-th label and the at least one first label to update the at least one n-th label.

### Claim 11

An image processing method, being adapted for use in an electronic apparatus, wherein the image processing method comprises the following steps:
downsizing a current frame and a reference frame to generate a down-sized current frame and a down-sized reference frame respectively;
marking at least one of a plurality of first blocks in the down-sized current frame as at least one first periodic pattern block with at least one first label respectively, wherein the at least one first periodic pattern block has a first periodic feature;
performing a first motion estimation on the down-sized current frame and the down-sized reference frame based on the at least one first label to generate a plurality of first motion vectors;
marking at least one of a plurality of n-th blocks in the current frame as at least one n-th periodic pattern block with at least one n-th label respectively, wherein the at least one n-th periodic pattern block has a n-th periodic feature;
performing an n-th motion estimation on the current frame and the reference frame based on the first motion vectors and the at least one n-th label to generate a plurality of n-th motion vectors; and
performing a motion compensation on the current frame and the reference frame based on the n-th motion vectors to generate a compensated frame between the current frame and the reference frame.

### Claim 12 (depends on 11)

The image processing method of claim 11, wherein the step of marking the at least one first periodic pattern block further comprises:
comparing a plurality of pixels and a plurality of shifted pixels in the down-sized current frame to calculate a plurality of differences, wherein the shifted pixels are the pixels after being shifted based on a plurality of different shifting values, and each of the differences is corresponding to one of the different shifting values; and
determining whether to mark the pixels as the at least one first periodic pattern block or not based on a minimum difference of the differences and a threshold.

### Claim 13 (depends on 12)

The image processing method of claim 12, wherein the step of marking the at least one first periodic pattern block further comprises:
calculating an interleaving frequency of the pixels corresponding to a mean value of the pixels; and
determining whether to mark the pixels as the at least one first periodic pattern block or not based on the minimum difference, the threshold, the interleaving frequency, and a most similar shifting value corresponding to the minimum difference.

### Claim 14 (depends on 12)

The image processing method of claim 12, wherein the first motion estimation comprises a plurality of scanning steps, and the scanning steps further comprises:
generating a searching window corresponding to each of the first blocks in the down-sized current frame;
adjusting the corresponding searching window based on the most similar shifting value and a shifting direction of each of the at least one first periodic pattern block;
generating a plurality of candidate vectors in the searching window; and
selecting the first motion vectors corresponding to each of the at least one first periodic pattern block from the candidate vectors.

### Claim 15 (depends on 11)

The image processing method of claim 11, wherein the n-th motion estimation comprises a plurality of a plurality of scanning steps, and the scanning steps further comprises:
generating a searching window corresponding to each of the n-th blocks in the current frame;
generating a plurality of candidate vectors in the searching window; and
selecting the n-th motion vectors corresponding to each of the at least one n-th periodic pattern block from the candidate vectors.

### Claim 16 (depends on 15)

The image processing method of claim 15, wherein the step of selecting one of the n-th motion vectors further comprises:
in response to the candidate vectors corresponding to one of the at least one n-th periodic pattern block, calculating a difference between a reference motion vector corresponding to the one of the at least one n-th periodic pattern block and each of the candidate vectors;
calculating a punish value of each of the candidate vectors based on the difference corresponding to each of the candidate vectors, wherein the difference and the punish value are positively related;
reducing a weight of each of the candidate vectors based on the punish value of each of the candidate vectors; and
selecting one of the n-th motion vectors from the candidate vectors based on the weight of each of the candidate vectors.

### Claim 17 (depends on 16)

The image processing method of claim 16, wherein the reference motion vector is one of a plurality of n−1-th motion vectors corresponding to the one of the at least one n-th periodic pattern block.

### Claim 18 (depends on 16)

The image processing method of claim 16, wherein the reference motion vector is a regional motion vector corresponding to the one of the at least one n-th periodic pattern block, wherein the regional motion vector is corresponding to the one of the at least one n-th periodic pattern block and at least one periodic pattern block nearby the one of the at least one n-th periodic pattern block.

### Claim 19 (depends on 16)

The image processing method of claim 16, wherein the step of selecting one of the n-th motion vectors further comprises:
in response to the candidate vectors corresponding to one of the at least one n-th periodic pattern block, removing a spatial candidate vector, a temporal candidate vector, and a random candidate vector from the candidate vectors; and
selecting one of the n-th motion vectors from the remaining candidate vectors.

### Claim 20 (depends on 11)

The image processing method of claim 11, wherein the step of marking the at least one of the n-th blocks in the current frame as the at least one n-th periodic pattern block with the at least one n-th label respectively further comprises:
performing an OR operation on the at least one n-th label and the at least one first label to update the at least one n-th label.
