Image enhancement method, image enhancement apparatus, and method and apparatus for training image enhancement apparatus

An image enhancement method comprising:

• generating an input raw image being generated by an under-display camera (UDC);
• acquiring a fifth intermediate vector by performing denoising on the input raw image using a denoising network;
• acquiring a sixth intermediate vector by performing deblurring on the fifth intermediate vector using a deblurring network;
• acquiring a fourth result vector for at least one channel by performing a noise compensation on the sixth intermediate vector based on a third residual signal between the input raw image and the fifth intermediate vector;
• and
• outputting an enhanced output image based on the fourth result vector for the at least one channel,
• wherein a first intermediate vector is acquired by performing denoising on a training raw image generated by the UDC, using the denoising network, a first result vector for the at least one channel is acquired by performing deblurring on the first intermediate vector using the deblurring network, a first loss value between the first result vector for the at least one channel and a training ground truth (GT) image corresponding to the training raw image is calculated using at least one first differentiator, and the deblurring network and the at least one first differentiator are trained based on the first loss value, and
• wherein a third intermediate vector is acquired by performing denoising on the training GT image using the denoising network, a second result vector for the at least one channel is acquired by performing blurring on the third intermediate vector using a blurring network, a second loss value between the second result vector for the at least one channel and the training raw image is calculated using at least one second differentiator, and the blurring network and the at least one second differentiator are trained based on the second loss value.

An apparatus for training an image enhancement apparatus, the apparatus comprising:

• an under-display camera (UDC) configured to generate a training raw image using;
• and
• a processor configured to:
• acquire a first intermediate vector by performing denoising on the training raw image using a denoising network;
• acquire a first result vector for at least one channel by performing deblurring on the first intermediate vector using a deblurring network;
• calculate a first loss value between the first result vector and a training ground truth (GT) image corresponding to the training raw image using at least one first differentiator, and train the deblurring network and the at least one first differentiator based on the first loss value;
• acquire a third intermediate vector by performing denoising on the training GT image using the denoising network;
• acquire a second result vector for the at least one channel by performing blurring on the third intermediate vector using a blurring network;
• and
• calculate a second loss value between the second result vector for the at least one channel and the training raw image using at least one second differentiator, and train the blurring network and the at least one second differentiator based on the second loss value.