# US 9,438,908 B1 · Motion vector interpolation based on true motion and layer information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,438,908 B1
- Document ID: US9438908B1
- Status: Granted
- Kind: B1
- Application number: 14/189,076
- Filed: 2014-02-25
- Granted: 2016-09-06
- Earliest claimed: 2014-02-25
- Assignees: Pixelworks, Inc.
- Applicants: Pixelworks, Inc.
- Inventors: Xiaojie Sheng; Neil Woodall; Bob Zhang; Guohua Cheng; Yue Ma
- Web page: https://patentgrep.com/patent/US9438908B1

## CPC

- H04N 19/52: by predictive encoding
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/513: Processing of motion vectors
- H04N 19/587: involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method to produce a frame of video data includes receiving original frame true motion information, original frame object boundary information, and original frame layer information for at least two original frames of video data, and generating interpolation phase block-level motion and interpolation phase pixel-level layer information from the original frame block-level true motion information, original frame pixel-level object boundary information, and original frame pixel-level layer information for the original frame.

## Claims

### Claim 1

A method to produce a frame of video data, comprising:
receiving original frame true motion information, original frame object boundary information, and original frame layer information for at least two original frames of video data; and
generating block boundary information and block level motion information for layer objects in an interpolated frame from the original frame block-level true motion information, original frame pixel-level object boundary information, and original frame pixel-level layer information for the original frame by projecting boundary pixels of the layer objects from the original frames into the interpolated frame;
retrieving pixel level layer information of the interpolated frame from the original frames by projecting the generated block level motion information of the layer objects of the interpolated frame into the original frames;
integrating the generated block level motion and pixel level layer information of the layer objects to generate a single interpolated frame of video data.

### Claim 2 (depends on 1)

The method of claim 1, further comprising filling hole regions at the interpolation phase.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating interpolation phase block level motion and interpolation phase pixel level layer information comprises:
projecting boundary pixels to the interpolated phase using original motion vectors;
determining if a block has at least one boundary pixel; and
if the block has at least one boundary pixel, identifying the block as a boundary block.

### Claim 4 (depends on 2)

The method of claim 2, wherein filling hole regions comprises identifying regions at the interpolation phase where there is no corresponding region from the original frames of video data.

### Claim 5 (depends on 4)

The method of claim 4, wherein filling hole regions at the interpolation phase comprises using a zero motion vector to point to the original frames of video data and retrieving pixel layer information.

### Claim 6 (depends on 4)

The method of claim 4, where in filling hole regions at the interpolation phase comprises receiving block motion inputs from a user.

### Claim 7 (depends on 1)

The method of claim 1, wherein using the block motion and pixel-level layer information to generate an interpolated frame of video data comprises interpolating motion and pixel information from a larger block to a sub-block.

### Claim 8 (depends on 7)

The method of claim 7, wherein interpolating motion and pixel information from a larger block to a sub-block comprises:
defining a mode that identifies motion to applied to the sub-block; and
defining a mask that designates application of the motion to the sub-block.

### Claim 9 (depends on 8)

The method of claim 8, wherein defining the mode comprises one of copying, interpolating and blending the motion.

### Claim 10 (depends on 9)

The method of claim 9, wherein defining the mode comprises copying and defining a mask comprises one of choosing a diagonal block, a vertical block, a horizontal block and a current block.

### Claim 11 (depends on 9)

The method of claim 9, wherein defining the mode comprises interpolating and defining a mask comprises using motion of a current block, linearly interpolating motion horizontally, linearly interpolating motion vertically, and bilinearly interpolating motion.

### Claim 12 (depends on 9)

The method of claim 9, wherein defining the mode comprises blending the motion and defining the mask comprises one of interpolation, interpolation weighted, different weighted and different.