Natural language processing machine learning frameworks trained using multi-task training routines

An apparatus for generating a cross-document similarity measure for an unlabeled document data object and a label document data object, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the processor, cause the apparatus to at least:

• generate, using an attention-based text encoder machine learning model and based at least in part on the unlabeled document data object, an unlabeled document word-wise embedded representation set for the unlabeled document data object that comprises one or more unlabeled document word-wise embedded representations for one or more unlabeled document words of the unlabeled document data object, wherein:
• (i) the attention-based text encoder machine learning model is configured to generate, for each input document data object, a respective word-wise embedded representation set that comprises one or more word-wise embedded representations for one or more words of the input document data object and (ii) generating the attention-based text encoder machine learning model comprises:
• identifying a group of training input document data objects comprising one or more training unlabeled document data objects and a plurality of label document data objects,
• identifying one or more training document pairs, wherein each training document pair comprises a respective training unlabeled document data object and a respective label document data object that is related to the respective training unlabeled document data object based at least in part on ground-truth cross-document relationships,
• for each training document pair, generating a cross-document distance measure,
• generating a concurrent learning loss model based at least in part on:
• (i) a language modeling loss model that is defined in accordance with a language modeling training task and based at least in part on the group of training input document data objects, and (ii) a similarity determination loss model that is determined based at least in part on each cross-document distance measure, and
• generating, using the concurrent learning loss model, one or more optimized parameter values for the attention-based text encoder machine learning model;
• generate, using the attention-based text encoder machine learning model and based at least in part on the label document data object, a label document word-wise embedded representation set for the label document data object that comprises one or more label document word-wise embedded representations for one or more label document words of the label document data object;
• generate, based at least in part on the one or more unlabeled document word-wise embedded representations and the one or more label document word-wise embedded representations, the cross-document similarity measure;
• and
• perform one or more prediction-based actions based at least in part on the cross-document similarity measure.

A computer program product for generating a cross-document similarity measure for an unlabeled document data object and a label document data object, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions configured to:

• generate, using an attention-based text encoder machine learning model and based at least in part on the unlabeled document data object, an unlabeled document word-wise embedded representation set for the unlabeled document data object that comprises one or more unlabeled document word-wise embedded representations for one or more unlabeled document words of the unlabeled document data object, wherein:
• (i) the attention-based text encoder machine learning model is configured to generate, for each input document data object, a respective word-wise embedded representation set that comprises one or more word-wise embedded representations for one or more words of the input document data object and (ii) generating the attention-based text encoder machine learning model comprises:
• identifying a group of training input document data objects comprising one or more training unlabeled document data objects and a plurality of label document data objects,
• identifying one or more training document pairs, wherein each training document pair comprises a respective training unlabeled document data object and a respective label document data object that is related to the respective training unlabeled document data object based at least in part on ground-truth cross-document relationships,
• for each training document pair, generating a cross-document distance measure,
• generating a concurrent learning loss model based at least in part on:
• (i) a language modeling loss model that is defined in accordance with a language modeling training task and based at least in part on the group of training input document data objects, and (ii) a similarity determination loss model that is determined based at least in part on each cross-document distance measure, and
• generating, using the concurrent learning loss model, one or more optimized parameter values for the attention-based text encoder machine learning model;
• generate, using the attention-based text encoder machine learning model and based at least in part on the label document data object, a label document word-wise embedded representation set for the label document data object that comprises one or more label document word-wise embedded representations for one or more label document words of the label document data object;
• generate, based at least in part on the one or more unlabeled document word-wise embedded representations and the one or more label document word-wise embedded representations, the cross-document similarity measure;
• and
• perform one or more prediction-based actions based at least in part on the cross-document similarity measure.