Back to results
US 2022/0092737 A1Published application
Video upsampling using one or more neural networks
AssigneeNVIDIA Corporation
InventorsShiqiu Liu, Matthieu Le, Andrew Tao
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques to enhance video are disclosed. In at least one embodiment, one or more neural networks are used to create a higher resolution video using upsampled frames from a lower resolution video.
Claims30
A graphics processor (GPU) comprising:
- a graphics core comprising:
- an instruction cache;
- a cache/shared memory;
- floating point logic units to perform 16-bit, 32-bit, and 64-bit floating point operations;
- integer logic units;
- and
- matrix processing units (MPUs) to perform half-precision floating point and 8-bit integer operations;
- memory;
- a memory controller;
- a Peripheral Component Interconnect express (PCIe) interface;
- a link coupled to a GPU-to-GPU bridge;
- logic to receive a low-resolution video frame including a low-resolution image and to up-sample the low-resolution image to generate a high-resolution image using a neural network;
- and
- a warp module to generate a warped history image;
- wherein the neural network is trained using pairs of images including an image to be up-sampled and a higher resolution image.
12 more elements
A system-on-chip (SOC), comprising:
- a graphics processor, comprising:
- a graphics core comprising:
- an instruction cache;
- a cache/shared memory;
- floating point logic units to perform 16-bit, 32-bit, and 64-bit floating point operations;
- integer logic units;
- and
- matrix processing units (MPUs) to perform half-precision floating point and 8-bit integer operations;
- memory;
- a memory controller;
- a Peripheral Component Interconnect express (PCIe) interface;
- a link coupled to a GPU-to-GPU bridge;
- logic to receive a low-resolution video frame including a low-resolution image and to up-sample the low-resolution image to generate a high-resolution image using a neural network;
- and
- a warp module to generate a warped history image;
- wherein the neural network is trained using pairs of images including an image to be up-sampled and a higher resolution image.
13 more elements
A method, comprising:
- receiving, by a graphics processor, a low-resolution video frame including a low-resolution image, wherein the graphics processor comprises:
- a graphics core comprising:
- an instruction cache;
- a cache/shared memory;
- floating point logic units to perform 16-bit, 32-bit, and 64-bit floating point operations;
- integer logic units;
- and
- matrix processing units (MPUs) to perform half-precision floating point and 8-bit integer operations;
- memory;
- a memory controller;
- a Peripheral Component Interconnect express (PCIe) interface;
- a link coupled to a GPU-to-GPU bridge;
- up-sampling, by the graphics processor, the low-resolution image to generate a high-resolution image using a neural network, wherein the neural network is trained using pairs of images including an image to be up-sampled and a higher resolution image;
- and
- warping, by the graphics processor, a history image.
12 more elements
Dependencies are inferred from the claim text.