# US 2025/0097381 A1 · Content-adaptive frame-rate upconversion for video coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0097381 A1
- Document ID: US20250097381A1
- Status: Published application
- Kind: A1
- Application number: 18/468,042
- Filed: 2023-09-15
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Nitin Bandwar; Wesley Elizaga Jonson; David Hansen; Alireza Shoa Hassani Lashdan
- Web page: https://patentgrep.com/patent/US20250097381A1

## CPC

- H04N 7/014: {involving the use of motion vectors (motion estimation and compensation in video coding H04N19/51)}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- H04N 7/0127: {by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter}
- H04N 7/0135: {involving interpolation processes (interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59)}
- G06T 5/40: using histogram techniques
- H04N 7/0137: {dependent on presence/absence of motion, e.g. of motion zones (H04N7/014 takes precedence; movement detection in television signals H04N5/144)}
- H04N 19/172: the region being a picture, frame or field
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An example device for upconverting video data includes a memory configured to store video data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: determine an amount of motion between a region of a first frame of the video data and a corresponding region of a second frame of the video data; determine a number of frames to interpolate between the first frame and the second frame according to the amount of motion; and interpolate the determined number of frames to upconvert the video data.

## Claims

### Claim 1

A method of upconverting video data, the method comprising:
determining an amount of motion between a region of a first frame of video data and a corresponding region of a second frame of the video data;
determining a number of frames to interpolate between the first frame and the second frame according to the amount of motion; and
interpolating the determined number of frames to upconvert the video data.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the amount of motion comprises performing a motion search to generate a motion vector for the region of the first frame such that the motion vector identifies the corresponding region of the second frame relative to the region of the first frame.

### Claim 3 (depends on 2)

The method of claim 2, wherein performing the motion search comprises:
performing the motion search on the first frame relative to the second frame to generate a plurality of motion vectors; and
determining that the region corresponds to a largest motion vector of the plurality motion vectors.

### Claim 4 (depends on 2)

The method of claim 2, wherein determining the amount of motion comprises:
performing the motion search on the first frame relative to the second frame to generate a plurality of motion vectors; and
generating a histogram including bins representing different lengths of motion vectors and values representing a number of the plurality of motion vectors included in each bin.

### Claim 5 (depends on 4)

The method of claim 4, wherein determining the number of frames to interpolate comprises determining the number of frames to interpolate according to a profile for the histogram.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining the number of frames to interpolate comprises:
comparing the amount of motion to a threshold; and
when the amount of motion is less than the threshold, determining a first number of frames to interpolate, or
when the amount of motion is greater than the threshold, determining a second number of frames to interpolate.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the number of frames to interpolate comprises:
determining a plurality of threshold values, each of the threshold values being different and being mapped to a respective number of frames to interpolate;
determining a largest threshold value of the plurality of threshold values that the amount of motion exceeds; and
determining that the number of frames to interpolate is equal to the number of frames to interpolate to which the largest threshold value that the amount of motion exceeds is mapped.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the number of frames to interpolate comprises:
when the amount of motion exceeds a first threshold but not a second threshold larger than the first threshold, determining the number of frames to interpolate is equal to 1,
when the amount of motion exceeds the second threshold but not a third threshold larger than the second threshold, determining the number of frames to interpolate is equal to 2, or
when the amount of motion exceeds the third threshold, determining the number of frames to interpolate is equal to 3.

### Claim 9 (depends on 1)

The method of claim 1, further comprising decoding the first frame and the second frame prior to determining the amount of motion.

### Claim 10 (depends on 1)

The method of claim 1, further comprising generating the first frame and the second frame prior to determining the amount of motion.

### Claim 11

A device for upconverting video data, the device comprising:
a memory configured to store video data; and
a processing system comprising one or more processors implemented in circuitry, the processing system being configured to:
determine an amount of motion between a region of a first frame of the video data and a corresponding region of a second frame of the video data;
determine a number of frames to interpolate between the first frame and the second frame according to the amount of motion; and
interpolate the determined number of frames to upconvert the video data.

### Claim 12 (depends on 11)

The device of claim 11, wherein to determine the amount of motion, the processing system is configured to perform a motion search to generate a motion vector for the region of the first frame such that the motion vector identifies the corresponding region of the second frame relative to the region of the first frame.

### Claim 13 (depends on 12)

The device of claim 12, wherein to performing the motion search, the processing system is configured to:
perform the motion search on the first frame relative to the second frame to generate a plurality of motion vectors; and
determine that the region corresponds to a largest motion vector of the plurality of motion vectors.

### Claim 14 (depends on 12)

The device of claim 12, wherein to performing the motion search, the processing system is configured to:
perform the motion search on the first frame relative to the second frame to generate a plurality of motion vectors; and
generate a histogram including bins representing different lengths of motion vectors and values representing a number of the plurality of motion vectors included in each bin.

### Claim 15 (depends on 11)

The device of claim 11, wherein to determine the number of frames to interpolate, the processing system is configured to:
compare the amount of motion to a threshold; and
when the amount of motion is less than the threshold, determine a first number of frames to interpolate, or
when the amount of motion is greater than the threshold, determine a second number of frames to interpolate.

### Claim 16 (depends on 11)

The device of claim 11, wherein to determine the number of frames to interpolate, the processing system is configured to:
determine a plurality of threshold values, each of the threshold values being different and being mapped to a respective number of frames to interpolate;
determine a largest threshold value of the plurality of threshold values that the amount of motion exceeds; and
determine that the number of frames to interpolate is equal to the number of frames to interpolate to which the largest threshold value that the amount of motion exceeds is mapped.

### Claim 17 (depends on 11)

The device of claim 11, wherein to determine the number of frames to interpolate, the processing system is configured to:
when the amount of motion exceeds a first threshold but not a second threshold larger than the first threshold, determine that the number of frames to interpolate is equal to 1,
when the amount of motion exceeds the second threshold but not a third threshold larger than the second threshold, determine that the number of frames to interpolate is equal to 2, or
when the amount of motion exceeds the third threshold, determine that the number of frames to interpolate is equal to 3.

### Claim 18 (depends on 11)

The device of claim 11, further comprising a video decoder configured to decode the first frame and the second frame.

### Claim 19 (depends on 11)

The device of claim 11, further comprising a graphics engine configured to generate the first frame and the second frame.

### Claim 20 (depends on 11)

The device of claim 11, further comprising a display configured to display the first frame, the second frame, and the interpolated frames.

### Claim 21 (depends on 11)

The device of claim 11, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

### Claim 22

A computer-readable storage medium having stored thereon instructions that, when executed, cause a processing system to:
determine an amount of motion between a region of a first frame of video data and a corresponding region of a second frame of the video data;
determine a number of frames to interpolate between the first frame and the second frame according to the amount of motion; and
interpolate the determined number of frames to upconvert the video data.

### Claim 23 (depends on 22)

The computer-readable storage medium of claim 22, wherein the instructions that cause the processing system to determine the amount of motion comprise instructions that cause the processing system to perform a motion search to generate a motion vector for the region of the first frame such that the motion vector identifies the corresponding region of the second frame relative to the region of the first frame.

### Claim 24 (depends on 23)

The computer-readable storage medium of claim 23, wherein the instructions that cause the processing system to perform the motion search comprise instructions that cause the processing system to:
perform the motion search on the entire first frame relative to the second frame to generate a plurality of motion vectors; and
determine that the motion vector for the region is a largest motion vector of the plurality of motion vectors.

### Claim 25 (depends on 23)

The computer-readable storage medium of claim 23, wherein the instructions that cause the processing system to perform the motion search comprise instructions that cause the processing system to:
perform the motion search on the first frame relative to the second frame to generate a plurality of motion vectors; and
generate a histogram including bins representing different lengths of motion vectors and values representing a number of the plurality of motion vectors included in each bin.

### Claim 26

A device for upconverting video data, the device comprising:
means for determining an amount of motion between a region of a first frame of video data and a corresponding region of a second frame of the video data;
means for determining a number of frames to interpolate between the first frame and the second frame according to the amount of motion; and
means for interpolating the determined number of frames to upconvert the video data.

### Claim 27 (depends on 26)

The device of claim 26, wherein the means for determining the amount of motion comprises means for performing a motion search to generate a motion vector for the region of the first frame such that the motion vector identifies the corresponding region of the second frame relative to the region of the first frame.

### Claim 28 (depends on 27)

The device of claim 27, wherein the means for performing the motion search comprises:
means for performing the motion search on the entire first frame relative to the second frame to generate a plurality of motion vectors; and
means for determining that the motion vector for the region is a largest motion vector of the plurality of motion vectors.

### Claim 29 (depends on 27)

The device of claim 27, wherein the means for performing the motion search comprises:
means for performing the motion search on the entire first frame relative to the second frame to generate a plurality of motion vectors; and
means for generating a histogram including bins representing different lengths of motion vectors and values representing a number of the plurality of motion vectors included in each bin.

### Claim 30 (depends on 26)

The device of claim 26, wherein the means for determining the number of frames to interpolate comprises:
means for comparing the amount of motion to a threshold; and
means for determining a first number of frames to interpolate when the amount of motion is less than the threshold; and
means for determining a second number of frames to interpolate when the amount of motion is greater than the threshold.
