# US 2023/0274490 A1 · Methods and systems for 0-phase motion vectors in 3d graphics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274490 A1
- Document ID: US20230274490A1
- Status: Published application
- Kind: A1
- Application number: 17/656,131
- Filed: 2022-03-23
- Published: 2023-08-31
- Earliest claimed: 2022-02-28
- Assignees: Pixelworks Semiconductor Technology (Shanghai) Co. Ltd.
- Applicants: Pixelworks Semiconductor Technology (Shanghai) Co., Ltd.
- Inventors: Hongmin Zhang; Miao Sima; Gongxian Liu; Zongming Han; Junhua Chen; Guohua Cheng; Baochen Liu; Neil Woodall; Yue Ma; Huili Han
- Web page: https://patentgrep.com/patent/US20230274490A1

## CPC

- G06T 15/04: Texture mapping
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/50: Depth or shape recovery
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 15/205: {Image-based rendering}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Various methods are provided for the generation of motion vectors in the context of 3D computer-generated images. In one example, a method includes generating, for each pixel of one or more objects to be rendered in a current frame, a 1-phase motion vector (MV1) and a 0-phase motion vector (MV0), each MV1 and MV0 having an associated depth value, to thereby form an MV1 texture and an MV0 texture, each MV0 determined based on a camera MV0 and an object MV0, converting the MV1 texture to a set of MV1 pixel blocks and converting the MV0 texture to a set of MV0 pixel blocks and outputting the set of MV1 pixel blocks and the set of MV0 pixel blocks for image processing.

## Claims

### Claim 1

A method, comprising:
generating, for each pixel of one or more objects to be rendered in a current frame, a 1-phase motion vector (MV1) and a 0-phase motion vector (MV0), each MV1 and MV0 having an associated depth value, to thereby form an MV1 texture and an MV0 texture, each MV0 determined based on a camera MV0 and an object MV0;
converting the MV1 texture to a set of MV1 pixel blocks and converting the MV0 texture to a set of MV0 pixel blocks; and
outputting the set of MV1 pixel blocks and the set of MV0 pixel blocks for image processing.

### Claim 2 (depends on 1)

The method of claim 1, wherein each MV1 represents a change in a position of a vertex at that pixel from the current frame to a previous frame, wherein each MV0 represents a change in a position of the vertex at that pixel from the previous frame to the current frame, wherein the camera MV0 represents a change in a position of the vertex at that pixel from the previous frame to the current due to a change in position or orientation of a virtual camera, and wherein the object MV0 represents a change in a position of the vertex at that pixel from the previous frame to the current due to a change in position of the object in a world space.

### Claim 3 (depends on 2)

The method of claim 2, further comprising, for a selected pixel, determining the camera MV0 for the selected pixel based on a world space position of the selected pixel's corresponding vertex in the previous frame.

### Claim 4 (depends on 3)

The method of claim 3, wherein determining the camera MV0 for the selected pixel based on the world space position of the selected pixel's corresponding vertex in the previous frame comprises:
determining an eye space position of the vertex based on a depth of the selected vertex in the previous frame and a ray pointing from the virtual camera to the selected vertex on a near clip of a homogenous clip space and applying an inversed view matrix from the previous frame to the eye space position to determine the world space position;
applying a first view matrix of the current frame to the world space position to generate a position of the selected vertex in the homogenous clip space;
applying a second view matrix of the previous frame to the world space position to generate a preposition of the selected vertex in the homogenous clip space; and
calculating the camera MV0 as a difference between the position and the preposition, in each of a vertical axis, a horizontal axis, and a depth axis.

### Claim 5 (depends on 2)

The method of claim 2, further comprising for a selected pixel, determining the object MV0 for the selected pixel based on an eye space position of a selected vertex corresponding to the selected pixel in the previous frame.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining the object MV0 for the selected pixel based on the eye space position of the selected vertex in the previous frame comprises:
calculating the eye space position of the selected vertex in the previous frame and calculating an eye space position of the selected vertex in the current frame, each based on a corresponding model matrix and view projection matrix;
interpolating the eye space position in the previous frame and the eye space position in the current frame into a homogenous clip space to generate an input position and an input preposition, respectively;
calculating the object MV0 as a difference between the input position and the input preposition, in each of a vertical axis, a horizontal axis, and a depth axis; and
assigning the object MV0 to the selected pixel based on the on-screen location of the selected vertex.

### Claim 7 (depends on 2)

The method of claim 2, wherein converting the MV1 texture to the set of MV1 pixel blocks comprises identifying, for each pixel block of a plurality of pixel blocks of the MV1 texture, a distribution of pixel types in that pixel block and converting the MV1 texture to the set of MV1 pixel blocks based on the distribution of pixel types for each pixel block.

### Claim 8 (depends on 7)

The method of claim 7, wherein converting the MV1 texture to the set of MV1 pixel blocks based on the distribution of pixel types for each pixel block comprises:
if the distribution of a selected pixel block is bimodal such that at least one background pixel and at least one foreground pixel are present in the selected pixel block, calculating an average MV1 over only foreground pixels in the selected pixel block, otherwise calculating the average MV1 over only mid-range pixels in the selected pixel block.

### Claim 9 (depends on 2)

The method of claim 2, wherein converting the MV0 texture to the set of MV0 pixel blocks comprises identifying, for each pixel block of a plurality of pixel blocks of the MV0 texture, a distribution of pixel types in that pixel block and converting the MV0 texture to the set of MV0 pixel blocks based on the distribution of pixel types for each pixel block.

### Claim 10 (depends on 9)

The method of claim 9, wherein converting the MV0 texture to the set of MV0 pixel blocks based on the distribution of pixel types for each pixel block comprises:
if the distribution of a selected pixel block is bimodal such that at least one background pixel and at least one foreground pixel are present in the selected pixel block, calculating an average MV0 over only foreground pixels in the selected pixel block, otherwise calculating the average MV0 over only mid-range pixels in the selected pixel block.

### Claim 11 (depends on 2)

The method of claim 2, further comprising applying a double-confirm process to each MV0 and each MV1, wherein the double-confirm process includes confirming each associated depth value.

### Claim 12 (depends on 11)

The method of claim 11, wherein applying the double-confirm process includes, for a selected pixel in the current frame, double confirming the MV1 or MV0 for the selected pixel responsive to coordinates of the selected pixel in the current frame matching coordinates of that pixel mapped back to the previous frame using the MV1 or MV0 for the selected pixel, the coordinates of the selected pixel including a depth coordinate.

### Claim 13

A system for rendering 3D graphics, comprising:
one or more processors and non-transitory memory allocated to form:
a motion vector module configured to receive 3D model information from a game engine and configured to output a plurality of 1-phase motion vectors (MV1s) and a plurality of 0-phase motion vectors (MV0s) based on the 3D model information, each MV1 comprising a change in a vertical position, a change in a horizontal position, and a change in a depth position of an object at a respective vertex from a current frame to a previous frame, each MV0 comprising a change in a vertical position, a change in a horizontal position, and a change in a depth position at a respective vertex from the previous frame to the current frame, wherein the motion vector module caches a subset of the 3D model information in order to calculate each MV0, including caching a depth buffer and an inversed view matrix from the previous frame; and
a block motion vector module configured to generate a block MV1 texture and a block MV0 texture from the plurality of MV1s and the plurality of MV0s, respectively, and output the block MV1 texture and the block MV0 texture for image processing in order to form an image to be displayed on a display, wherein the block MV1 texture comprises a plurality of MV1 blocks each formed from a respective subset of the plurality of MV1s, and wherein the block MV0 texture comprises a plurality of MV0 blocks each formed from a respective subset of the plurality of MV0s.

### Claim 14 (depends on 13)

The system of claim 13, wherein the block MV1 texture and the block MV0 texture are usable to interpolate a frame between the current frame and the previous frame.

### Claim 15 (depends on 13)

The system of claim 13, wherein each MV0 is determined based on a respective camera MV0 and a respective object MV0, where a camera MV0 for a selected vertex is determined based on a world space position of the selected vertex in the previous frame determined based on the cached depth buffer and inversed view matrix, where an object MV0 for the selected vertex is determined based on an eye space position of the selected vertex in the previous frame and an eye space position of the selected vertex in the current frame.

### Claim 16 (depends on 13)

The system of claim 13, where each MV1 is assigned a respective pixel based on a location of a corresponding vertex.

### Claim 17 (depends on 13)

The system of claim 13, where each MV0 is assigned a respective pixel based on a location of a corresponding vertex.

### Claim 18 (depends on 13)

The system of claim 13, wherein each MV1 block comprises an average change in horizontal position for that respective subset of the plurality of MV1s, an average change in vertical position for that respective subset of the plurality of MV1s, and an average depth value for that respective subset of the plurality of MV1 s, and wherein each MV0 block comprises an average change in horizontal position for that respective subset of the plurality of MV0s, an average change in vertical position for that respective subset of the plurality of MV0s, and an average depth value for that respective subset of the plurality of MV0s.

### Claim 19

A method, comprising:
receiving, from a game engine, 3D model information usable to render a first current frame on a 2D screen;
caching a subset of the 3D model information, including caching a depth texture and an inversed view projection matrix;
receiving, from the game engine, updated 3D model information usable to render a second current frame on the 2D screen, including an updated depth texture and an updated inversed view projection matrix;
calculating a plurality of 0-phase motion vectors from the updated 3D model information and the cached depth texture and inversed view projection matrix;
outputting the plurality of 0-phase motion vectors as block 0-phase motion vectors for image processing.

### Claim 20 (depends on 19)

The method of claim 19, wherein the 3D model information includes the first current frame and the updated 3D model information includes the second current frame, and further comprising calculating a plurality of 1-phase motion vectors from the updated 3D model information, the updated depth texture, and the inversed view projection matrix; and
outputting the plurality of 1-phase motion vectors as block 1-phase motion vectors for image processing.
