Natural language processing machine learning frameworks trained using multi-task training routines

An apparatus for generating a cross-document similarity measure for an unlabeled document data object and a label document data object, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the processor, cause the apparatus to at least:

• generate, using an attention-based text encoder machine learning model and based at least in part on the unlabeled document data object, an unlabeled document word-wise embedded representation set for the unlabeled document data object, wherein:
• (i) the attention-based text encoder machine learning model is configured to generate, for each input document data object, a respective document-wide embedded representation and a respective word-wise embedded representation set, and (ii) generating the attention-based text encoder machine learning model comprises:
• identifying a group of training input document data objects comprising one or more training unlabeled document data objects and a plurality of label document data objects,
• identifying one or more training document pairs, wherein each training document pair comprises a respective training unlabeled document data object and a respective label document data object that is related to the respective training unlabeled document data object based at least in part on ground-truth cross-document relationships,
• generating, using a language modeling loss model that is defined in accordance with a language modeling training task and based at least in part on the group of training input document data objects, one or more initially-optimized parameter values for one or more trainable parameters of the attention-based text encoder machine learning model,
• for each training document pair, generating a cross-document distance measure based at least in part on the respective document-wide embedded representation for the respective training unlabeled document data object and the respective label document data object,
• generating a similarity determination loss model based at least in part on each cross-document distance measure,
• generating, using a sequential learning loss model that is determined based at least in part on adjusting the similarity determination loss model in accordance with a sequential learning regularization factor that describes computed effects of potential updates to the one or more initially-optimized parameter values on the language modeling loss model, one or more subsequently-optimized parameter values for the one or more trainable parameters of the attention-based text encoder machine learning model;
• generate, using the attention-based text encoder machine learning model and based at least in part on the label document data object, a label document word-wise embedded representation set for the unlabeled document data object;
• generate, based at least in part on the unlabeled document word-wise embedded representation set and the label document word-wise embedded representation set, the cross-document similarity measure;
• and
• perform one or more prediction-based actions based at least in part on the cross-document similarity measure.

A computer program product for generating a cross-document similarity measure for an unlabeled document data object and a label document data object, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions configured to:

• generate, using an attention-based text encoder machine learning model and based at least in part on the unlabeled document data object, an unlabeled document word-wise embedded representation set for the unlabeled document data object, wherein:
• (i) the attention-based text encoder machine learning model is configured to generate, for each input document data object, a respective document-wide embedded representation and a respective word-wise embedded representation set, and (ii) generating the attention-based text encoder machine learning model comprises:
• identifying a group of training input document data objects comprising one or more training unlabeled document data objects and a plurality of label document data objects,
• identifying one or more training document pairs, wherein each training document pair comprises a respective training unlabeled document data object and a respective label document data object that is related to the respective training unlabeled document data object based at least in part on ground-truth cross-document relationships,
• generating, using a language modeling loss model that is defined in accordance with a language modeling training task and based at least in part on the group of training input document data objects, one or more initially-optimized parameter values for one or more trainable parameters of the attention-based text encoder machine learning model,
• for each training document pair, generating a cross-document distance measure based at least in part on the respective document-wide embedded representation for the respective training unlabeled document data object and the respective label document data object,
• generating a similarity determination loss model based at least in part on each cross-document distance measure,
• generating, using a sequential learning loss model that is determined based at least in part on adjusting the similarity determination loss model in accordance with a sequential learning regularization factor that describes computed effects of potential updates to the one or more initially-optimized parameter values on the language modeling loss model, one or more subsequently-optimized parameter values for the one or more trainable parameters of the attention-based text encoder machine learning model;
• generate, using the attention-based text encoder machine learning model and based at least in part on the label document data object, a label document word-wise embedded representation set for the unlabeled document data object;
• generate, based at least in part on the unlabeled document word-wise embedded representation set and the label document word-wise embedded representation set, the cross-document similarity measure;
• and
• perform one or more prediction-based actions based at least in part on the cross-document similarity measure.