Annotating and collecting data-centric ai quality metrics considering user preferences

A computer system for collecting and annotating data based on user preference, the computer system comprising:

• one or more computer-readable non-transitory storage media configured to store computer program code;
• and
• one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:
• receiving code configured to cause the one or more computer processors to receive unlabeled data corresponding to one or more entries within a dataset;
• generating code configured to cause the one or more computer processors to generate pseudo-labeled data based on the unlabeled data;
• and
• determining code configured to cause the one or more computer processors to determine, based on one or more quality metrics, each entry from among the pseudo-labeled data is to be included within a final dataset, wherein the one or more entries are selected by two or more hidden layers corresponding to, at least, an input layer and an output, wherein the input layer corresponds to a feature map of input samples and the output layer corresponds to a probability of considering each entry for inclusion in the final dataset.

A non-transitory computer readable medium having stored thereon a computer program for collecting and annotating data based on user preference, the computer program configured to cause one or more computer processors to:

• receive unlabeled data corresponding to one or more entries within a dataset;
• generate pseudo-labeled data based on the unlabeled data;
• and
• determine, based on one or more quality metrics, each entry from among the pseudo-labeled data is to be included within a final dataset, wherein the one or more entries are selected by two or more hidden layers corresponding to, at least, an input layer and an output, wherein the input layer corresponds to a feature map of input samples and the output layer corresponds to a probability of considering each entry for inclusion in the final dataset.