Natural language processing machine learning frameworks trained using multi-task training routines

An apparatus for generating a document classification for an unlabeled document data object, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the processor, cause the apparatus to at least:

• generate, using an attention-based text encoder machine learning model and based at least in part on the unlabeled document data object, an unlabeled document word-wise embedded representation set for the unlabeled document data object, wherein:
• (i) the attention-based text encoder machine learning model is configured to generate, for each input document data object, a respective word-wise embedded representation set, and (ii) generating the attention-based text encoder machine learning model comprises:
• identifying one or more training input document data objects each associated with a ground-truth label classification,
• generating, using a language modeling loss model that is defined in accordance with a language modeling training task and based at least in part on the one or more training input document data objects, one or more initially-optimized parameter values for one or more trainable parameters of the attention-based text encoder machine learning model,
• for each training input document data object:
• generating, using the attention-based text encoder machine learning model, the respective word-wise embedded representation set for the training input document data object, and
• generating, using a document classification machine learning model and based at least in part on the respective word-wise embedded representation set for the training input document data object, an inferred document classification for the training input document data object, and
• generating, based at least in part on the inferred document classification for the training input document data object and a ground-truth document classification for the training input document data object, a per-document classification loss model for the training input document data object,
• generating a document classification loss model based at least in part on each per-document classification loss model, and
• generating, using a sequential learning loss model that is determined based at least in part on adjusting the document classification loss model in accordance with a sequential learning regularization factor that describes computed effects of potential updates to the one or more initially-optimized parameter values on the language modeling loss model, one or more subsequently-optimized parameter values for the attention-based text encoder machine learning model;
• generate, using the document classification machine learning model and based at least in part on the unlabeled document document-wide embedded representation, the document classification;
• and
• perform one or more prediction-based actions based at least in part on the document classification.

A computer program product for generating a document classification for an unlabeled document data object, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions configured to:

• generate, using an attention-based text encoder machine learning model and based at least in part on the unlabeled document data object, an unlabeled document word-wise embedded representation set for the unlabeled document data object, wherein:
• (i) the attention-based text encoder machine learning model is configured to generate, for each input document data object, a respective word-wise embedded representation set, and (ii) generating the attention-based text encoder machine learning model comprises:
• identifying one or more training input document data objects each associated with a ground-truth label classification,
• generating, using a language modeling loss model that is defined in accordance with a language modeling training task and based at least in part on the one or more training input document data objects, one or more initially-optimized parameter values for one or more trainable parameters of the attention-based text encoder machine learning model,
• for each training input document data object:
• generating, using the attention-based text encoder machine learning model, the respective word-wise embedded representation set for the training input document data object, and
• generating, using a document classification machine learning model and based at least in part on the respective word-wise embedded representation set for the training input document data object, an inferred document classification for the training input document data object, and
• generating, based at least in part on the inferred document classification for the training input document data object and a ground-truth document classification for the training input document data object, a per-document classification loss model for the training input document data object,
• generating a document classification loss model based at least in part on each per-document classification loss model, and
• generating, using a sequential learning loss model that is determined based at least in part on adjusting the document classification loss model in accordance with a sequential learning regularization factor that describes computed effects of potential updates to the one or more initially-optimized parameter values on the language modeling loss model, one or more subsequently-optimized parameter values for the attention-based text encoder machine learning model;
• generate, using the document classification machine learning model and based at least in part on the unlabeled document document-wide embedded representation, the document classification;
• and
• perform one or more prediction-based actions based at least in part on the document classification.