Systems and methods for machine-learned prediction of semantic similarity between documents

A computing system for training a machine-learned model for semantic document analysis, comprising:

• one or more processors;
• and
• a memory, comprising:
• a machine-learned semantic document encoding transformer model comprising a first encoding submodel and a second encoding submodel, each of the first and second encoding submodels comprising a sentence encoding portion and a document encoding portion, wherein:
• the sentence encoding portion is configured to process a plurality of textual blocks to obtain a plurality of textual block representations;
• and
• the document encoding portion is configured to process the plurality of textual block representations to obtain a plurality of contextual block representations;
• and
• one or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising:
• obtaining a first document comprising a plurality of first sentences and a second document comprising a plurality of second sentences;
• parsing the first document into a plurality of first textual blocks and the second document into a plurality of second textual blocks, wherein each of the plurality of first textual blocks comprises one or more of the plurality of first sentences and each of the plurality of second textual blocks comprises one or more of the plurality of second sentences, wherein each of the plurality of first textual blocks comprises a textual capacity, and wherein parsing the first document into the plurality of first textual blocks comprises:
• determining that a sentence parsed from the plurality of first sentences to a textual block of the plurality of first textual blocks would exceed the textual capacity of the textual block;
• and
• parsing the sentence to a second textual block of the plurality of first textual blocks;
• processing the plurality of first textual blocks and the plurality of second textual blocks with the machine-learned semantic document encoding transformer model to respectively obtain a plurality of first contextual block representations and a plurality of second contextual block representations;
• determining, based on at least one of the plurality of first contextual block representations and at least one of the plurality of second contextual block representations, a similarity metric descriptive of a semantic similarity between the first document and the second document;
• evaluating a loss function that evaluates a difference between the similarity metric and ground truth data associated with the first document and the second document;
• and
• adjusting one or more parameters of the machine-learned semantic document encoding transformer model based at least in part on the loss function.

One or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

• obtaining a first document comprising a plurality of first sentences and a second document comprising a plurality of second sentences;
• parsing the first document into a plurality of first textual blocks and the second document into a plurality of second textual blocks, wherein each of the plurality of first textual blocks comprises one or more of the plurality of first sentences and each of the plurality of second textual blocks comprises one or more of the plurality of second sentences, wherein each of the plurality of first textual blocks comprises a textual capacity, and wherein parsing the first document into the plurality of first textual blocks comprises:
• determining that a sentence parsed from the plurality of first sentences to a textual block of the plurality of first textual blocks would exceed the textual capacity of the textual block;
• and
• parsing the sentence to a second textual block of the plurality of first textual blocks;
• processing each of the plurality of first textual blocks with a block encoding portion of a first encoding submodel of a machine-learned semantic document encoding transformer model to obtain a respective plurality of first textual block representations;
• processing each of the plurality of second textual blocks with a block encoding portion of a second encoding submodel of the machine-learned semantic document encoding transformer model to obtain a respective plurality of second textual block representations;
• respectively processing the plurality of first textual block representations and the plurality of second textual block representations with a document encoding portion of the first encoding submodel and a document encoding portion of the second encoding submodel to obtain a first document encoding and a second document encoding;
• and
• determining a similarity metric descriptive of a semantic similarity between the first document and the second document based on the first document encoding and the second document encoding.