# US 2026/0179177 A1 · Image generation method and apparatus based on video frame interpolation, and computer device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0179177 A1
- Document ID: US20260179177A1
- Status: Published application
- Kind: A1
- Application number: 19/538,818
- Filed: 2026-02-12
- Published: 2026-06-25
- Earliest claimed: 2023-11-14
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Hu YE
- Web page: https://patentgrep.com/patent/US20260179177A1

## CPC

- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- H04N 7/014: {involving the use of motion vectors (motion estimation and compensation in video coding H04N19/51)}
- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06T 7/248: {involving reference images or patches}
- G06T 7/269: using gradient-based methods
- H04N 7/0127: {by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter}
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

This application relates to a video frame interpolation method performed by a computer device. The method includes: determining first optical flow information between two video frame images; predicting an intermediate frame image between the two video frame images based on the first optical flow information; for each of the two video frame images, determining second optical flow information between the video frame image and the predicted intermediate frame image; selecting image blocks corresponding to the first pixel points from the video frame image according to the second optical flow information; updating pixel values at first pixel points in the predicted intermediate frame image based on image blocks corresponding to the first pixel points in the predicted intermediate frame image; and fusing updated intermediate frame images respectively corresponding to the two video frame images to obtain a fused intermediate frame image.

## Claims

### Claim 1

A video frame interpolation method performed by a computer device, the method comprising:
determining first optical flow information between two video frame images according to displacement of pixel points in the two video frame images;
predicting an intermediate frame image between the two video frame images based on the first optical flow information;
for each of the two video frame images;
determining second optical flow information between the video frame image and the predicted intermediate frame image according to displacement of pixel points in the video frame image and the predicted intermediate frame image;
selecting, for each first pixel point in the predicted intermediate frame image, an image block corresponding to the first pixel point from the video frame image according to the second optical flow information; and
updating pixel values at first pixel points in the predicted intermediate frame image based on image blocks corresponding to the first pixel points in the predicted intermediate frame image to obtain an updated intermediate frame image corresponding to the video frame image; and
fusing updated intermediate frame images respectively corresponding to the two video frame images to obtain a fused intermediate frame image for interpolation between the two video frame images.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the updating pixel values at first pixel points in the predicted intermediate frame image based on image blocks corresponding to the first pixel points in the predicted intermediate frame image to obtain an updated intermediate frame image corresponding to the video frame image comprises:
acquiring, for each first pixel point in the predicted intermediate frame image, position information of the image block corresponding to the first pixel point from the video frame image; and
updating the pixel value at the first pixel point based on the image block corresponding to the first pixel point and the position information to obtain an updated pixel value at the first pixel point to obtain the updated intermediate frame image corresponding to the video frame image.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the image block corresponding to the first pixel point comprises a plurality of second pixel points, and the position information comprises respective pixel point positions of the plurality of second pixel points; and
the updating the pixel value at the first pixel point based on the image block corresponding to the first pixel point and the position information to obtain an updated pixel value at the first pixel point to obtain the updated intermediate frame image corresponding to the video frame image comprises:
performing, for each of the plurality of second pixel points, feature extraction based on a pixel value at the second pixel point and the pixel point position of the second pixel point to obtain pixel point features corresponding to the second pixel point; and
updating the pixel value at the first pixel point based on pixel point features respectively corresponding to the plurality of second pixel points to obtain the updated pixel value at the first pixel point.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the performing feature extraction based on a pixel value at the second pixel point and the pixel point position of the second pixel point to obtain pixel point features corresponding to the second pixel point comprises:
encoding the pixel value at the second pixel point to obtain a pixel value code corresponding to the second pixel point, and encoding the pixel point position of the second pixel point to obtain a position code corresponding to the second pixel point; and
fusing the pixel value code and the position code that correspond to the second pixel point to obtain the pixel point features corresponding to the second pixel point.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the updating the pixel value at the first pixel point based on pixel point features respectively corresponding to the plurality of second pixel points to obtain the updated pixel value at the first pixel point comprises:
performing feature extraction based on the pixel value at the first pixel point and the position offset corresponding to the first pixel point to obtain pixel point features corresponding to the first pixel point; and
performing feature interaction on the pixel point features respectively corresponding to the plurality of second pixel points and the pixel point features corresponding to the first pixel point based on an attention mechanism to obtain the updated pixel value at the first pixel point.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the fusing updated intermediate frame images respectively corresponding to the two video frame images to obtain a fused intermediate frame image for interpolation between the two video frame images comprises:
performing feature extraction on the updated intermediate frame images respectively corresponding to the two video frame images to obtain updated intermediate frame image features respectively corresponding to the two video frame images;
fusing the updated intermediate frame image features respectively corresponding to the two video frame images to obtain fused features; and
performing feature mapping based on the fused features to obtain the fused intermediate frame image for interpolation between the two video frame images.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the performing feature mapping based on the fused features to obtain the fused intermediate frame image for interpolation between the two video frame images comprises:
performing feature mapping on pixel point features of a plurality of pixel points in the fused features to obtain pixel values at third pixel points in the fused intermediate frame image; and
obtaining, according to the pixel values at the third pixel points, the fused intermediate frame image for interpolation between the two video frame images.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the selecting an image block corresponding to the first pixel point from the video frame image according to the second optical flow information comprises:
determining a position offset corresponding to the first pixel point from the second optical flow information;
determining a first pixel position corresponding to the first pixel point from the video frame image according to the position offset; and
selecting the image block corresponding to the first pixel point from the video frame image based on the first pixel position.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the predicting an intermediate frame image between the two video frame images based on the two video frame images and the first optical flow information comprises:
estimating intermediate frame optical flow information respectively corresponding to the two video frame images based on forward optical flow information and backward optical flow information in the first optical flow information and an intermediate frame moment; and
predicting the intermediate frame image between the two video frame images based on the two video frame images and the intermediate frame optical flow information respectively corresponding to the two video frame images to obtain the predicted intermediate frame image.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the predicting the intermediate frame image between the two video frame images based on the two video frame images and the intermediate frame optical flow information respectively corresponding to the two video frame images to obtain the predicted intermediate frame image comprises:
predicting, for each of the two video frame images, the intermediate frame image between the two video frame images based on the video frame image and the intermediate frame optical flow information corresponding to the video frame image to obtain a predicted image corresponding to the video frame image; and
fusing predicted images respectively corresponding to the two video frame images to obtain the predicted intermediate frame image.

### Claim 11

A computer device, comprising a memory and a processor, the memory having a computer program stored therein, and the computer program, when executed by the processor, causing the computer device to implement a video frame interpolation method including:
determining first optical flow information between two video frame images according to displacement of pixel points in the two video frame images;
predicting an intermediate frame image between the two video frame images based on the first optical flow information;
for each of the two video frame images;
determining second optical flow information between the video frame image and the predicted intermediate frame image according to displacement of pixel points in the video frame image and the predicted intermediate frame image;
selecting, for each first pixel point in the predicted intermediate frame image, an image block corresponding to the first pixel point from the video frame image according to the second optical flow information; and
updating pixel values at first pixel points in the predicted intermediate frame image based on image blocks corresponding to the first pixel points in the predicted intermediate frame image to obtain an updated intermediate frame image corresponding to the video frame image; and
fusing updated intermediate frame images respectively corresponding to the two video frame images to obtain a fused intermediate frame image for interpolation between the two video frame images.

### Claim 12 (depends on 11)

The computer device according to claim 11, wherein the updating pixel values at first pixel points in the predicted intermediate frame image based on image blocks corresponding to the first pixel points in the predicted intermediate frame image to obtain an updated intermediate frame image corresponding to the video frame image comprises:
acquiring, for each first pixel point in the predicted intermediate frame image, position information of the image block corresponding to the first pixel point from the video frame image; and
updating the pixel value at the first pixel point based on the image block corresponding to the first pixel point and the position information to obtain an updated pixel value at the first pixel point to obtain the updated intermediate frame image corresponding to the video frame image.

### Claim 13 (depends on 12)

The computer device according to claim 12, wherein the image block corresponding to the first pixel point comprises a plurality of second pixel points, and the position information comprises respective pixel point positions of the plurality of second pixel points; and
the updating the pixel value at the first pixel point based on the image block corresponding to the first pixel point and the position information to obtain an updated pixel value at the first pixel point to obtain the updated intermediate frame image corresponding to the video frame image comprises:
performing, for each of the plurality of second pixel points, feature extraction based on a pixel value at the second pixel point and the pixel point position of the second pixel point to obtain pixel point features corresponding to the second pixel point; and
updating the pixel value at the first pixel point based on pixel point features respectively corresponding to the plurality of second pixel points to obtain the updated pixel value at the first pixel point.

### Claim 14 (depends on 13)

The computer device according to claim 13, wherein the performing feature extraction based on a pixel value at the second pixel point and the pixel point position of the second pixel point to obtain pixel point features corresponding to the second pixel point comprises:
encoding the pixel value at the second pixel point to obtain a pixel value code corresponding to the second pixel point, and encoding the pixel point position of the second pixel point to obtain a position code corresponding to the second pixel point; and
fusing the pixel value code and the position code that correspond to the second pixel point to obtain the pixel point features corresponding to the second pixel point.

### Claim 15 (depends on 13)

The computer device according to claim 13, wherein the updating the pixel value at the first pixel point based on pixel point features respectively corresponding to the plurality of second pixel points to obtain the updated pixel value at the first pixel point comprises:
performing feature extraction based on the pixel value at the first pixel point and the position offset corresponding to the first pixel point to obtain pixel point features corresponding to the first pixel point; and
performing feature interaction on the pixel point features respectively corresponding to the plurality of second pixel points and the pixel point features corresponding to the first pixel point based on an attention mechanism to obtain the updated pixel value at the first pixel point.

### Claim 16 (depends on 11)

The computer device according to claim 11, wherein the fusing updated intermediate frame images respectively corresponding to the two video frame images to obtain a fused intermediate frame image for interpolation between the two video frame images comprises:
performing feature extraction on the updated intermediate frame images respectively corresponding to the two video frame images to obtain updated intermediate frame image features respectively corresponding to the two video frame images;
fusing the updated intermediate frame image features respectively corresponding to the two video frame images to obtain fused features; and
performing feature mapping based on the fused features to obtain the fused intermediate frame image for interpolation between the two video frame images.

### Claim 17 (depends on 11)

The computer device according to claim 11, wherein the selecting an image block corresponding to the first pixel point from the video frame image according to the second optical flow information comprises:
determining a position offset corresponding to the first pixel point from the second optical flow information;
determining a first pixel position corresponding to the first pixel point from the video frame image according to the position offset; and
selecting the image block corresponding to the first pixel point from the video frame image based on the first pixel position.

### Claim 18 (depends on 11)

The computer device according to claim 11, wherein the predicting an intermediate frame image between the two video frame images based on the two video frame images and the first optical flow information comprises:
estimating intermediate frame optical flow information respectively corresponding to the two video frame images based on forward optical flow information and backward optical flow information in the first optical flow information and an intermediate frame moment; and
predicting the intermediate frame image between the two video frame images based on the two video frame images and the intermediate frame optical flow information respectively corresponding to the two video frame images to obtain the predicted intermediate frame image.

### Claim 19 (depends on 18)

The computer device according to claim 18, wherein the predicting the intermediate frame image between the two video frame images based on the two video frame images and the intermediate frame optical flow information respectively corresponding to the two video frame images to obtain the predicted intermediate frame image comprises:
predicting, for each of the two video frame images, the intermediate frame image between the two video frame images based on the video frame image and the intermediate frame optical flow information corresponding to the video frame image to obtain a predicted image corresponding to the video frame image; and
fusing predicted images respectively corresponding to the two video frame images to obtain the predicted intermediate frame image.

### Claim 20

A non-transitory computer-readable storage medium having a computer program stored therein, and the computer program, when executed by a processor of a computer device, causing the computer device to implement a video frame interpolation method including:
determining first optical flow information between two video frame images according to displacement of pixel points in the two video frame images;
predicting an intermediate frame image between the two video frame images based on the first optical flow information;
for each of the two video frame images;
determining second optical flow information between the video frame image and the predicted intermediate frame image according to displacement of pixel points in the video frame image and the predicted intermediate frame image;
selecting, for each first pixel point in the predicted intermediate frame image, an image block corresponding to the first pixel point from the video frame image according to the second optical flow information; and
updating pixel values at first pixel points in the predicted intermediate frame image based on image blocks corresponding to the first pixel points in the predicted intermediate frame image to obtain an updated intermediate frame image corresponding to the video frame image; and
fusing updated intermediate frame images respectively corresponding to the two video frame images to obtain a fused intermediate frame image for interpolation between the two video frame images.
