Systems, apparatus, and methods for super-resolution of non-uniform blur
AssigneeGoPro, Inc.
InventorsCharles Laroche, Matias Tassano Ferres, Andrés Almansa
Life of this patent
Estimating the term…
Abstract
Systems, apparatus, and methods for super-resolution of non-uniform spatial blur. Non-uniform spatial blur presents unique challenges for conventional neural network processing. Existing implementations attempt to handle super-resolution with a “brute force” optimization. Various embodiments of the present disclosure subdivide the super-resolution function into sub-steps. “Unfolding” super-resolution into smaller closed-form functions allows for operation using generic plug-and-play convolutional neural network (CNN) logic. Additionally, each step can be optimized with its own step-specific hyper parameters to improve performance.
Claims20
A method, comprising:
- obtaining an image and a blurred version;
- iteratively performing a super-resolution function based on a deconvolution of a non-uniform spatial blur on the image over a plurality of iterations, the super-resolution function comprising at least a prior stage of a linearized alternating direction method of multipliers (ADMM), a data stage of the linearized ADMM, and an update stage of the linearized ADMM, where:
- during the prior stage, configuring a convolutional neural network processor to denoise the blurred version into a denoised version based on a regularization parameter;
- during the data stage, configuring the convolutional neural network processor to compute a proximal operator of the blurred version and the denoised version based on a penalty parameter;
- and
- during the update stage, calculating an update parameter used to configure the convolutional neural network processor for a next iteration of the prior stage and the data stage based on the denoised version and the proximal operator;
- and outputting a super-resolution image.
An apparatus, comprising:
- a processor;
- a convolutional neural network;
- and
- a non-transitory computer-readable medium comprising instructions that when executed by the processor, causes the processor to:
- obtain an image;
- and
- iteratively resolve the image with the convolutional neural network over a plurality of iterations according to a super-resolution function, where each iteration of the plurality of iterations comprises a prior stage of a linearized alternating direction method of multipliers (ADMM), a data stage of the linearized ADMM, and an update stage of the linearized ADMM.
A non-transitory computer-readable medium comprising one or more instructions which, when executed by a processor, causes the processor to configure a convolutional neural network to iteratively:
- denoise a blurred version of an image into a denoised version during a prior stage of a linearized alternating direction method of multipliers (ADMM);
- compute a proximal operator of the image and the denoised version during a data stage of the linearized ADMM;
- and
- update an update parameter used to configure the convolutional neural network for a next iteration of the prior stage and the data stage based on the denoised version and the proximal operator during an update stage of the linearized ADMM.
Dependencies are inferred from the claim text.