Image restoration for through-display imaging
AssigneeMicrosoft Technology Licensing, LLC
InventorsYuqian Zhou, Timothy Andrew Large, Se Hoon Lim, Neil Emerton, Yonghuan David Ren
Life of this patent
Estimating the term…
Abstract
Examples are disclosed that relate to the restoration of degraded images acquired via a behind-display camera. One example provides a method of training a machine learning model, the method comprising inputting training image pairs into the machine learning model, each training image pair comprising an undegraded image and a degraded image that represents an appearance of the undegraded image to a behind-display camera, and training the machine learning model using the training image pairs to generate frequency information that is missing from the degraded images.
Claims7
A method of training a machine learning model, the method comprising:
- inputting training image pairs into the machine learning model, the machine learning model comprising a U-shaped neural network comprising a first sub-encoder and a second sub-encoder in parallel with the first sub-encoder, each training image pair comprising an undegraded image and a degraded image that represents an appearance of the undegraded image to a behind-display camera, the degraded image comprising loss of mid-frequency information due to diffraction-related blurring, the mid-frequency information within a range of 2 to 8 cycles per degree;
- and
- training the machine learning model using the training image pairs to generate mid-frequency information to correct for the loss of mid-frequency information due to the diffraction-related blurring, wherein the first sub-encoder is trained to compute residual details and the second sub-encoder is trained to learn content encoding.
Dependencies are inferred from the claim text.