Modeling for indexing and semiconductor defect image retrieval

A method, comprising:

• receiving, by a processing device, a first image frame indicative of a substrate processing defect;
• generating, by the processing device, a second image frame by cropping a first region of the first image frame;
• generating, by the processing device, a third image frame by cropping a second region of the first image frame, wherein the first region comprises the second region;
• using the second image frame as input to a first machine learning (ML) model;
• obtaining one or more outputs of the first ML model, the one or more outputs indicating a first feature vector corresponding to the second image frame;
• using the third image frame as input to a second ML model;
• obtaining one or more outputs of the second ML model, the one or more outputs indicating a second feature vector corresponding to the third image frame;
• updating one or more parameters of at least one of the first ML model or the second ML model based on a comparison between the one or more outputs of the first ML model and the one or more outputs of the second ML model.

A non-transitory machine-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

• store, in a data storage device, a plurality of feature vectors representative of previously processed image frames that correspond to various substrate processing defects;
• receive first image data comprising one or more image frames indicative of a first substrate processing defect;
• determine a first feature vector corresponding to the first image data;
• determine a selection of the plurality of feature vectors based on a proximity between the first feature vector and each of the selection of the plurality of feature vectors;
• determine second image data comprising one or more image frames corresponding to the selection of the plurality of embedding vectors;
• perform an action based on determining the second image data.