Device and method for defect inspection based on explainable artificial intelligence

A system for inspecting a defect, comprising:

• a deep learning-based first classification model and a deep learning-based second classification model, wherein each of the first classification model and the second classification model includes an encoder, an explainable artificial intelligence (XAI), and a fully-connected layer, the encoder is configured to generate a latent vector from the image, the XAI is configured to generate the determination basis from the latent vector, and the fully-connected layer is configured to generate the determination result from the latent vector;
• an input unit configured to acquire an image of a product to be inspected;
• a product inspection unit configured to generate, using the deep first classification model, a determination result indicating whether the product is good or defective based on the acquired image, and to provide a determination basis for the first classification model to generate the determination result using the XAI to generate a category set for the determination basis, wherein the first classification model has one or more parameters of a deep learning algorithm of the first classification model that are same as one or more parameters of the second classification model that is pre-trained or stored in a data storage after being updated;
• and
• an adaptation unit configured to perform an adaptation process on the first classification model when the determination basis is determined as being an abnormal value for adapting the first classification model to the abnormal value by updating the parameters of the deep learning algorithm of the first classification model using the XAI to clarify the determination basis for determining whether the product is good or defective,
• wherein the adaptation unit comprises:
• a category set generation unit configured to update the category set including a plurality of categories for the determination basis;
• and
• a training unit configured to train the first classification model using the second classification model and the updated category set, and
• further wherein:
• the training unit is configured to perform a pre-training on the second classification model using seen data for training and a corresponding target determination result,
• the category set generation unit is configured to initialize the category set by classifying the plurality of categories for the determination basis generated by the second classification model using the seen data used for performing the pre-training,
• updating the category set comprises:
• classifying, as unseen data, an image causing the first classification model to generate the abnormal value and the corresponding determination result;
• and
• causing the determination result corresponding to the abnormal value to be included in the corresponding target determination result,
• training the first classification model comprises:
• performing a latent space matching using the seen data;
• and
• performing a fine-tuning using the unseen data,
• the latent space matching and fine-tuning are repeatedly performed on the first classification model until the performance of the first classification model satisfies a preset reference performance,
• performing the latent space matching includes updating the one or more parameters of the first classification model to reduce a first loss function, the first loss function being defined based on a difference between the determination basis generated by the first classification model using the seen data and the determination basis generated by the second classification model using the seen data, and
• performing the fine-tuning comprises updating the one or more parameters of the first classification model to reduce a second loss function, the second loss function being defined based on a difference between the determination result generated by the first classification model using the unseen data and the corresponding target determination result.

A non-transitory computer-readable recording medium storing instructions that, when executed by a processor, cause a processor to control a system to perform:

• a deep learning-based first classification model and a deep learning-based second classification model, wherein each of the first classification model and the second classification model includes an encoder, an explainable artificial intelligence (XAI), and a fully-connected layer, the encoder is configured to generate a latent vector from the image, the XAI is configured to generate the determination basis from the latent vector, and the fully-connected layer is configured to generate the determination result from the latent vector;
• acquiring an image of a product to be inspected;
• generating, using the first classification model, a determination result indicating whether the product is good or defective based on the acquired image;
• providing a determination basis for the first classification model to generate the determination result using the XAI to generate a category set for the determination basis, wherein the first classification model has one or more parameters of a deep learning algorithm of the first classification model that are same as one or more parameters of the second classification model that is pre-trained or stored in a data storage after being updated;
• and
• performing an adaptation process on the first classification model when the determination basis is determined as being an abnormal value for adapting the first classification model to the abnormal value by updating the parameters of the deep learning algorithm of the first classification model using the XAI to clarify the determination basis for determining whether the product is good or defective,
• wherein performing the adaptation process comprises:
• updating the category set including a plurality of categories for the determination basis;
• and
• training the first classification model using the second classification model and the updated category set, and
• further wherein:
• the second classification model is pre-trained using seen data for training and a corresponding target determination result,
• the category set is initialized by classifying the plurality of categories for the determination basis generated by the second classification model using the seen data,
• updating the category set comprises:
• classifying, as unseen data, an image causing the first classification model to generate the abnormal value and the corresponding determination result;
• and
• causing the determination result corresponding to the abnormal value to be included in the corresponding target determination result,
• training the first classification model comprises:
• performing a latent space matching using the seen data;
• and
• performing a fine-tuning using the unseen data,
• the latent space matching and fine-tuning are repeatedly performed on the first classification model until the performance of the first classification model satisfies a preset reference performance,
• performing the latent space matching includes updating the one or more parameters of the first classification model to reduce a first loss function, the first loss function being defined based on a difference between the determination basis generated by the first classification model using the seen data and the determination basis generated by the second classification model using the seen data, and
• performing the fine-tuning comprises updating the one or more parameters of the first classification model to reduce a second loss function, the second loss function being defined based on a difference between the determination result generated by the first classification model using the unseen data and the corresponding target determination result.