Image synthesis with feature-level supervision and per-step optimization

#This disclosure relates generally to computer vision and anomaly detection, and more particularly to digital image synthesis via a diffusion-based machine learning model with feature-level supervision and per-step optimization.

#Synthesizing novel defects of manufacturing parts helps to build intelligent and robust machine learning models to detect defects when deployed on-line onto production assembly lines. Pretrained Diffusion models have been shown to synthesize realistic images. However, directly using them to synthesize various defects of specialized manufacturing parts poses challenges due to the specificity and complexity of such items, as well as some proprietary concerns.

#The following is a summary of certain embodiments described in detail below. The described aspects are presented merely to provide the reader with a brief summary of these certain embodiments and the description of these aspects is not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects that may not be explicitly set forth below.

#FIG. 1 is a flow diagram that provides an overview of an example of an image synthesis process via a machine learning model according to at least one example embodiment of this disclosure.

#FIG. 2A and FIG. 2B illustrate enlarged views of the non-limiting examples of the digital images of FIG. 1 according to at least one example embodiment of this disclosure.

#FIG. 3 illustrates an example of a process for finetuning a U-Net of FIG. 1 according to at least one embodiment of this disclosure.

#The embodiments described herein, which have been shown and described by way of example, and many of their advantages will be understood by the foregoing description, and it will be apparent that various changes can be made in the form, construction, and arrangement of the components without departing from the disclosed subject matter or without sacrificing one or more of its advantages. Indeed, the described forms of these embodiments are merely explanatory. These embodiments are susceptible to various modifications and alternative forms, and the following claims are intended to encompass and include such changes and not be limited to the particular forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling with the spirit and scope of this disclosure.

#Recent advances in deep learning have opened new possibilities for image synthesis and image editing. However, there has been little exploration into applying these technologies for the synthesis of specific defects at specified locations of digital images relating to some fields (e.g., manufacturing field, medical field, etc.). In particular, the challenge remains in accurately capturing defect patterns from one image and transferring them onto objects in digital images while maintaining realism and precision in the new image.

#As an example, in the field of manufacturing, ensuring the quality of produced parts is crucial to maintaining operational efficiency and product reliability. Defects in manufactured parts can lead to significant losses, both in terms of material wastage and time spent in manual inspection and correction. This also has serious implications for the safe usage of the manufactured goods and the reputation of the company. Traditional methods of identifying and simulating defects in manufacturing processes often rely on either manual inspection or machine vision systems that are limited by their ability to synthesize or transfer specific defect characteristics from one part to another. These limitations make it challenging to fully test the robustness of manufacturing systems and processes against a wide variety of defect types.