Methods and systems for training artificial intelligence-based models using limited labeled data

A server system, comprising:

• a communication interface;
• a memory comprising machine-readable instructions;
• and
• a processor communicably coupled to the communication interface and the memory, the processor configured to execute the machine-readable instructions to cause the server system at least in part to:
• access a tabular dataset from a database associated with the server system, the tabular dataset comprising tabular data related to a plurality of entities, the tabular data comprising labeled data and unlabeled data;
• generate a set of labeled features based, at least in part, on the labeled data, the set of labeled features comprising a set of labeled numerical features and a set of labeled categorical features;
• generate a set of unlabeled features based, at least in part, on the unlabeled data, the set of unlabeled features comprising a set of unlabeled numerical features and a set of unlabeled categorical features;
• determine, by the server system via a first transformer model, a set of contextual numerical embeddings based, at least in part, on the set of labeled numerical features and the set of unlabeled numerical features;
• determine, by the server system via a second transformer model, a set of contextual categorical embeddings based, at least in part, on the set of labeled categorical features and the set of unlabeled categorical features;
• generate a set of concatenated embeddings based, at least in part, on concatenating the set of contextual numerical embeddings and the set of contextual categorical embeddings;
• and
• generate a third transformer model based, at least in part, on the set of concatenated embeddings.

A non-transitory computer-readable storage medium comprising computer-executable instructions that, when executed by at least a processor of a server system, cause the server system to perform a method comprising:

• accessing a tabular dataset from a database associated with the server system, the tabular dataset comprising tabular data related to a plurality of entities, the tabular data comprising labeled data and unlabeled data;
• generating a set of labeled features based, at least in part, on the labeled data, the set of labeled features comprising a set of labeled numerical features and a set of labeled categorical features;
• generating a set of unlabeled features based, at least in part, on the unlabeled data, the set of unlabeled features comprising a set of unlabeled numerical features and a set of unlabeled categorical features;
• determining, via a first transformer model, a set of contextual numerical embeddings based, at least in part, on the set of labeled numerical features and the set of unlabeled numerical features;
• determining, via a second transformer model, a set of contextual categorical embeddings based, at least in part, on the set of labeled categorical features and the set of unlabeled categorical features;
• generating a set of concatenated embeddings based, at least in part, on concatenating the set of contextual numerical embeddings and the set of contextual categorical embeddings;
• and
• generating a third transformer model based, at least in part, on the set of concatenated embeddings.