Learning method and learning device for reducing distortion occurred in warped image generated in process of stabilizing jittered image by using GAN to enhance fault tolerance and fluctuation robustness in extreme situations

A testing method for testing reduction of distortion occurred in a warped image generated in a process of stabilizing a jittered image by using a GAN(Generative Adversarial Network) including a generating network and a discriminating network, comprising a step of:

• on condition that (1) a learning device has instructed an adjusting layer included in the generating network to adjust at least part of initial feature values for training corresponding to pixels included in at least one initial training image, to thereby transform the initial training image into at least one adjusted training image, and (2) the learning device has instructed a loss layer included in the generating network to generate a generating network loss by referring to at least part of (i) at least one naturality score representing at least one probability of the adjusted training image being determined as real by the discriminating network, (ii) at least one maintenance score representing whether one or more initial objects included in the initial training image are included in the adjusted training image with their characteristics maintained, and (iii) at least one similarity score representing a degree of similarity between the initial training image and the adjusted training image, and learn parameters of the generating network by backpropagating the generating network loss;
• a testing device instructing the adjusting layer included in the generating network to adjust at least part of initial feature values for testing corresponding to pixels included in at least one initial test image, to thereby transform the initial test image into at least one adjusted test image.

A learning device for learning reduction of distortion occurred in a warped image generated in a process of stabilizing a jittered image by using a GAN (Generative Adversarial Network) including a generating network and a discriminating network, comprising:

• at least one memory that stores instructions;
• and
• at least one processor configured to execute the instructions to:
• perform processes of (1) if at least one initial image is acquired, instructing an adjusting layer included in the generating network to adjust at least part of initial feature values corresponding to pixels included in the initial image, to thereby transform the initial image into at least one adjusted image, and (II) if at least part of (i) at least one naturality score representing at least one probability of the adjusted image being determined as real by the discriminating network, (ii) at least one maintenance score representing whether one or more initial objects included in the initial image are included in the adjusted image with their characteristics maintained, and (iii) at least one similarity score representing a degree of similarity between the initial image and the adjusted image are acquired, instructing a loss layer included in the generating network to generate a generating network loss by referring to said at least part of the naturality score, the maintenance score and the similarity score, and learn parameters of the generating network by backpropagating the generating network loss.

A testing device for reduction of distortion occurred in a warped image generated in a process of stabilizing a jittered image by using a GAN (Generative Adversarial Network) including a generating network and a discriminating network, comprising:

• at least one memory that stores instructions;
• and
• at least one processor, on condition that (1) a learning device has instructed an adjusting layer included in the generating network to adjust at least part of initial feature values for training corresponding to pixels included in at least one initial training image, to thereby transform the initial training image into at least one adjusted training image, and (2) the learning device has instructed a loss layer included in the generating network to generate a generating network loss by referring to at least part of (i) at least one naturality score representing at least one probability of the adjusted training image being determined as real by the discriminating network, (ii) at least one maintenance score representing whether one or more initial objects included in the initial training image are included in the adjusted training image with their characteristics maintained, and (iii) at least one similarity score representing a degree of similarity between the initial training image and the adjusted training image, and learn parameters of the generating network by backpropagating the generating network loss;
• configured to execute the instructions to:
• perform a process of instructing the adjusting layer included in the generating network to adjust at least part of initial feature values for testing corresponding to pixels included in at least one initial test image, to thereby transform the initial test image into at least one adjusted test image.