# US 2025/0166121 A1 · Image rendering method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166121 A1
- Document ID: US20250166121A1
- Status: Published application
- Kind: A1
- Application number: 19/023,129
- Filed: 2025-01-15
- Published: 2025-05-22
- Earliest claimed: 2022-10-20
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Shaobo Feng; Chang Liu; Shu Wang; Tianji Zheng
- Web page: https://patentgrep.com/patent/US20250166121A1

## CPC

- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 15/005: {General purpose rendering architectures}
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/50: Depth or shape recovery
- G06T 15/00: Three-dimensional [3D] image rendering
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image rendering method includes: rendering a first data frame at a first resolution to obtain a first image frame, where the first data frame is a 1st data frame of an application scene of an application that is running on an electronic device; rendering a plurality of second data frames at a second resolution to obtain a plurality of second image frames, where the second resolution is less than the first resolution, and the plurality of second data frames are subsequent data frames of the application scene; and compensating for rendering data of each of the plurality of second image frames by using the first image frame as a reference frame.

## Claims

### Claim 1

An image rendering method, comprising:
rendering, by an electronic device, a first data frame at a first resolution to obtain a first image frame, wherein the first data frame is a first data frame of a first scene of an application that is running on the electronic device;
rendering, by the electronic device, a plurality of second data frames at a second resolution to obtain a plurality of second image frames, wherein the second resolution is less than the first resolution, wherein the plurality of second data frames are data frames of the first scene that follow the first data frame, and wherein the plurality of second data frames have a one-to-one correspondence with the plurality of second image frames; and
performing, by the electronic device, a compensation process on rendering data of the plurality of second image frames based on using the first image frame as a reference frame.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the electronic device comprises a first buffer and a second buffer, wherein a size of the first buffer corresponds to the first resolution, and wherein a size of the second buffer corresponds to the second resolution;
wherein the first buffer is configured to buffer the reference frame, and wherein the second buffer is configured to buffer α respective second image frame of the plurality of second image frames; and
wherein performing the compensation process comprises performing the compensation process on rendering data of the respective second image frame in the second buffer based on rendering data of the reference frame in the first buffer.

### Claim 3 (depends on 2)

The method according to claim 2, wherein before rendering the first data frame to obtain the first image frame, the method further comprises:
obtaining scene indication information corresponding to the first data frame; and
determining that a change in application scene has occurred based on the scene indication information.

### Claim 4 (depends on 2)

The method according to claim 2, wherein before rendering the plurality of second data frames to obtain the plurality of second image frames, the method further comprises:
for a respective second data frame, obtaining scene indication information corresponding to the respective second data frame; and
determining that a change in application scene has not occurred based on the scene indication information.

### Claim 5 (depends on 2)

The method according to claim 2, wherein performing the compensation process comprises:
upsampling the respective second image frame in the second buffer to obtain an upsampled first temporary image;
reprojecting the reference frame in the first buffer relative to the respective second image frame to obtain a first reprojected image; and
performing compensation on rendering data of a pixel at a position in the first temporary image using rendering data of a pixel at a corresponding position in the first reprojected image.

### Claim 6 (depends on 5)

The method according to claim 5, wherein performing compensation on the rendering data of the pixel in the first temporary image comprises:
determining a similarity between a first pixel block and a second pixel block, wherein the first pixel block is a pixel block of N*M pixels in the first reprojected image, wherein the second pixel block is a pixel block of N*M pixels in the first temporary image, wherein a position of the first pixel block in the first reprojected image corresponds to a position of the second pixel block in the first temporary image, and wherein at least one of N and M is an integer greater than 1; and
based on the similarity being greater than a first threshold, replacing, with rendering data of a valid pixel in the first pixel block, rendering data of a pixel that is in the second pixel block and whose position corresponds to a position of the valid pixel, wherein the valid pixel is a pixel for which an absolute value of a difference between a first depth value and a second depth value is less than the first threshold, wherein the first depth value is a depth value of a pixel in the first reprojected image, and wherein the second depth value is a depth value of a pixel in the first temporary image.

### Claim 7 (depends on 6)

The method according to claim 6, wherein reprojecting the reference frame in the first buffer relative to the respective second image frame to obtain the first reprojected image comprises:
determining, based on coordinates of each pixel in the reference frame, a first product, and a second product, coordinates of each corresponding pixel obtained by reprojecting the reference frame relative to the respective second image frame, wherein the first product is a product of a view matrix and a projection matrix of the reference frame, and wherein the second product is a product of a view matrix and a projection matrix of the respective second image frame; and
obtaining the first reprojected image based on a comparison result between a depth value of each pixel obtained through reprojection and a depth value of a corresponding pixel in the first temporary image.

### Claim 8 (depends on 7)

The method according to claim 7, wherein obtaining the first reprojected image based on the comparison result between the depth value of each pixel obtained through reprojection and the depth value of the corresponding pixel in the first temporary image comprises:
obtaining an absolute value of a difference between a second depth value of each pixel in the first temporary image and a depth value of a corresponding pixel obtained through reprojection; and
based on the absolute value of the difference being less than a second threshold, identifying the pixel obtained through reprojection as a valid pixel, to obtain the first reprojected image.

### Claim 9 (depends on 2)

The method according to claim 2, wherein performing the compensation process comprises:
determining a residual image of the reference frame in the first buffer;
determining a second reprojected image of the residual image; and
performing compensation on rendering data of the respective second image frame in the second buffer based on the second reprojected image.

### Claim 10 (depends on 9)

The method according to claim 9, wherein determining the residual image of the reference frame in the first buffer comprises:
downsampling the reference frame in the first buffer to the second resolution and then performing upsampling, to obtain a second temporary image; and
determining the residual image of the reference frame based on the reference frame in the first buffer and the second temporary image.

### Claim 11 (depends on 9)

The method according to claim 9, wherein determining the second reprojected image of the residual image comprises:
determining, based on coordinates of each pixel in the residual image, a first product, and a second product, coordinates of each pixel obtained by reprojecting the residual image relative to the respective second image frame, to obtain the second reprojected image, wherein the first product is a product of a view matrix and a projection matrix of the reference frame, and wherein the second product is a product of a view matrix and a projection matrix of the respective second image frame.

### Claim 12

An electronic device, comprising:
a transceiver;
a processor; and
a memory;
wherein the transceiver and the processor are coupled to the memory;
wherein the memory is configured to store a program or instructions; and
wherein the processor is configured to execute the program or instructions to facilitate the following being performed by the electronic device:
rendering a first data frame at a first resolution to obtain a first image frame, wherein the first data frame is a first data frame of a first scene of an application that is running on the electronic device;
rendering a plurality of second data frames at a second resolution to obtain a plurality of second image frames, wherein the second resolution is less than the first resolution, wherein the plurality of second data frames are data frames of the first scene that follow the first data frame, and wherein the plurality of second data frames have a one-to-one correspondence with the plurality of second image frames; and
performing a compensation process on rendering data of the plurality of second image frames based on using the first image frame as a reference frame.

### Claim 13 (depends on 12)

The electronic device according to claim 12, wherein the electronic device comprises a first buffer and a second buffer, wherein a size of the first buffer corresponds to the first resolution, and wherein a size of the second buffer corresponds to the second resolution;
wherein the first buffer is configured to buffer the reference frame, and the second buffer is configured to buffer α respective second image frame of the plurality of second image frames; and
wherein the processor is configured to execute the program or instructions to facilitate the following being performed by the electronic device:
performing the compensation process on rendering data of the respective second image frame in the second buffer based on rendering data of the reference frame in the first buffer.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein the processor is further configured to execute the program or instructions to facilitate the following being performed by the electronic device:
obtaining scene indication information corresponding to the first data frame; and
determining that a change in application scene has occurred based on the scene indication information.

### Claim 15 (depends on 13)

The electronic device according to claim 13, wherein the processor is further configured to execute the program or instructions to facilitate the following being performed by the electronic device:
for the respective second data frame, obtaining scene indication information corresponding to the respective second data frame; and
determining that a change in application scene has not occurred based on the scene indication information.

### Claim 16 (depends on 13)

The electronic device according to claim 13, wherein the processor is further configured to execute the program or instructions to facilitate the following being performed by the electronic device:
upsampling the respective second image frame in the second buffer to obtain an upsampled first temporary image;
reprojecting the reference frame in the first buffer relative to the respective second image frame to obtain a first reprojected image; and
performing compensation on rendering data of a pixel at a position in the first temporary image using rendering data of a pixel at a corresponding position in the first reprojected image.

### Claim 17 (depends on 16)

The electronic device according to claim 16, wherein the processor is further configured to execute the program or instructions to facilitate the following being performed by the electronic device:
determining a similarity between a first pixel block and a second pixel block, wherein the first pixel block is a pixel block of N*M pixels in the first reprojected image, wherein the second pixel block is a pixel block of N*M pixels in the first temporary image, wherein a position of the first pixel block in the first reprojected image corresponds to a position of the second pixel block in the first temporary image, and wherein at least one of N and M is an integer greater than 1; and
based on the similarity being greater than a first threshold, replacing, with rendering data of a valid pixel in the first pixel block, rendering data of a pixel that is in the second pixel block and whose position corresponds to a position of the valid pixel, wherein the valid pixel is a pixel for which an absolute value of a difference between a first depth value and a second depth value is less than the first threshold, wherein the first depth value is a depth value of a pixel in the first reprojected image, and wherein the second depth value is a depth value of a pixel in the first temporary image.

### Claim 18 (depends on 17)

The electronic device according to claim 17, wherein the processor is further configured to execute the program or instructions to facilitate the following being performed by the electronic device:
determining, based on coordinates of each pixel in the reference frame, a first product, and a second product, coordinates of each corresponding pixel obtained by reprojecting the reference frame relative to the any one second image frame, wherein the first product is a product of a view matrix and a projection matrix of the reference frame, and wherein the second product is a product of a view matrix and a projection matrix of the respective second image frame; and
obtaining the first reprojected image based on a comparison result between a depth value of each pixel obtained through reprojection and a depth value of a corresponding pixel in the first temporary image.

### Claim 19 (depends on 13)

The electronic device according to claim 13, wherein the processor is further configured to execute the program or instructions to facilitate the following being performed by the electronic device:
determining a residual image of the reference frame in the first buffer;
determining a second reprojected image of the residual image; and
performing compensation on rendering data of the respective second image frame in the second buffer based on the second reprojected image.

### Claim 20

A non-transitory computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by an electronic device, facilitate performance of the following by the electronic device:
rendering a first data frame at a first resolution to obtain a first image frame, wherein the first data frame is a first data frame of a first scene of an application that is running on the electronic device;
rendering a plurality of second data frames at a second resolution to obtain a plurality of second image frames, wherein the second resolution is less than the first resolution, wherein the plurality of second data frames are data frames of the first scene that follow the first data frame, and wherein the plurality of second data frames have a one-to-one correspondence with the plurality of second image frames; and
performing a compensation process on rendering data of the plurality of second image frames based on using the first image frame as a reference frame.
