Systems and methods for multi-task baseline models in semiconductor metrology and inspection applications

A system comprising at least one processor and instructions executable by the at least one processor to cause the at least one processor to perform operations comprising:

• (a) receiving a first dataset comprising one or more unknown objects and a second dataset comprising a set of reference objects, wherein the set of reference objects comprises a set of reference embeddings associated with instances of a class of at least one unknown object or the reference objects;
• (b) generating a set of input embeddings of the one or more unknown objects by applying an encoder to the first dataset;
• (c) generating a set of correlated embeddings of the one or more unknown objects by applying a convolutional neural network (CNN) model to the set of input embeddings and the set of reference embeddings, wherein the set of correlated embeddings is used to generate at least a detection overlay of the one or more unknown objects;
• (d) generating a set of referenced-attentioned embeddings of the one or more unknown objects by applying a referenced-attentioned model to the set of input embeddings and the set of reference embeddings, wherein the set of referenced-attentioned embeddings is used to generate at least a segmentation overlay of the one or more unknown objects;
• and
• (e) training a machine learning (ML) model using at least one of the detection overlay or the segmentation overlay, wherein the trained ML model is used to predict at least one of a set of detection characteristics or a set of segmentation characteristics with a confidence score.