System and method for generating derained image using self-supervised learning model

A method for generating a derained image using a self-supervised learning model, comprising:

• processing, using a machine learning model, an input image received from a user device for classifying pixels of the input image, wherein the input image comprises at least one rainy pixel, wherein the machine learning model is implemented on a processor of an image deraining server;
• classifying, using the machine learning model, the pixels of the input image into the at least one rainy pixel or non-rainy pixel by assigning a first value to the at least one rainy pixel and a second value to the non-rainy pixel based on a pixel intensity, wherein the machine learning model is trained by providing (i) historical rainy images and corresponding non-rainy images and (ii) historical values of pixel intensities of the historical rainy images and corresponding non-rainy images that maps to the at least one rainy pixel or the non-rainy pixel based on a threshold value of the pixel intensity;
• predicting, using a probability estimation network (PEN), a pixel-wise rain likelihood in the input image, instead of learning non-rainy pixels to determine at least one rainy pixel in the input image, wherein the PEN is trained by differentiating the at least one rainy pixel and the non-rainy pixel in the input image based on the threshold value of the pixel intensity;
• removing, using a self-supervised learning model, and a prior knowledge of rain distribution, the at least one rainy pixel of an inpainted image, and image artifacts and blurriness introduced by an image inpainting method;
• generating, using the machine learning model, a masked area in the input image by masking out the at least one rainy pixel from the input image based on the first value;
• generating, using the machine learning model, an inpainted image by filling the masked area using the image inpainting method, wherein the inpainted image acts as the prior knowledge;
• and
• generating, using the machine learning model, an optimized image by removing the at least one rainy pixel of the inpainted image, image artifacts and blurriness caused by the at least one rainy pixel, wherein the optimized image enables an improved semantic segmentation of the input image and an object detection.

One or more non-transitory computer readable storage mediums configured with instructions executable by one or more processors to cause the one or more processors to perform a method of generating a derained image using a self-supervised learning model, comprising:

• processing, using a machine learning model, an input image received from a user device for classifying pixels of the input image, wherein the input image comprises at least one rainy pixel, wherein the machine learning model is implemented on a processor of an image deraining server;
• classifying, using the machine learning model, the pixels of the input image into the at least one rainy pixel or non-rainy pixel by assigning a first value to the at least one rainy pixel and a second value to the non-rainy pixel based on a pixel intensity, wherein the machine learning model is trained by providing (i) historical rainy images and corresponding non-rainy images and (ii) historical values of intensities of pixels of the historical rainy images and corresponding non-rainy images that maps to the rainy pixels or non-rainy pixels based on a threshold value of the pixel intensity, wherein the trained machine learning model comprises a Probability Estimation Network (PEN) that differentiates the rainy pixels and non-rainy pixels in the input image based on the threshold value, a prior knowledge generation model and the self-supervised learning model;
• predicting, using the PEN, a pixel-wise rain likelihood in the input image, instead of learning non-rainy pixel to determine the at least one rainy pixel in the input image;
• removing, using a self-supervised learning model using the prior knowledge, of at least one rainy pixel of an inpainted image, and image artifacts and blurriness introduced by an image inpainting method;
• generating, using the machine learning model, a masked area in the input image by masking out the at least one rainy pixel from the input image based on the first value;
• generating, using the machine learning model, the inpainted image by filling the masked area using the image inpainting method, wherein the inpainted image acts as the prior knowledge;
• and
• generating, using the machine learning model, an optimized image by removing the at least one rainy pixel of the inpainted image and remove image artifacts and blurriness caused by the at least one rainy pixel, wherein the optimized image enables an improved semantic segmentation of the input image and an object detection.