Video enhancement using a recurrent image date of a neural network
AssigneeAmazon Technologies, Inc.
InventorsNataliya SHAPOVALOVA, Walterio Wolfgang MAYOL CUEVAS, Silviu Stefan ANDREI
Life of this patent
Estimating the term…
Abstract
Techniques for enhancing an image are described. For example, a lower-resolution image from a video file may be enhanced using a trained neural network applying the trained neural network on the lower-resolution image to remove artifacts by generating, using a layer of the trained neural network, a residual value based on the proper subset of the received image and at least one corresponding image portion of a previously generated higher resolution image in the video file, upscaling the received image using bilinear upsampling, and combining the upscaled received image and residual value to generate an enhanced image.
Claims20
A computer-implemented method comprising:
- receiving a request to enhance one or more frames of a streaming video file;
- receiving a lower-resolution frame of the video to be at least partially enhanced using a trained neural network;
- performing object recognition on the lower-resolution frame to determine a proper subset of the lower-resolution frame to enhance;
- enhancing the proper subset of the lower-resolution frame using the trained neural network according to the request by:
- removing artifacts by generating, using a layer of the trained neural network, a residual value based on the proper subset of the received frame and at least one corresponding frame portion of a previously generated higher resolution frame in the video file,
- upscaling at least the proper subset of the received frame using bilinear upsampling, and
- combining the upscaled proper subset of the received image and residual value to generate an enhanced frame;
- and
- outputting the enhanced frame.
A computer-implemented method comprising:
- receiving a request to enhance one or more images of a video file;
- receiving a lower-resolution image of the video to be at least partially enhanced using a trained neural network;
- enhancing the lower-resolution image using the trained neural network according to the request by applying the trained neural network on the lower-resolution image to:
- removing artifacts by generating, using a layer of the trained neural network, a residual value based on the proper subset of the received image and at least one corresponding image portion of a previously generated higher resolution image in the video file,
- upscaling the received image using bilinear upsampling, and
- combining the upscaled received image and residual value to generate an enhanced image;
- and
- outputting the enhanced image.
A system comprising:
- storage for a video file;
- an image enhancement service implemented by one or more electronic devices, the image enhancement service including instructions that upon execution cause the image enhancement service to:
- receive a request to enhance one or more images of a video file,
- receive a lower-resolution image of the video to be at least partially enhanced using a trained neural network,
- enhance the lower-resolution image using the trained neural network according to the request by applying the trained neural network on the lower-resolution image to:
- remove artifacts by generating, using a layer of the trained neural network, a residual value based on the proper subset of the received image and at least one corresponding image portion of a previously generated higher resolution image in the video file,
- upscale the received image using bilinear upsampling, and
- combine the upscaled received image and residual value to generate an enhanced image, and
- output the enhanced image.
Dependencies are inferred from the claim text.