Graph path prediction and masked language modelling joint training algorithm for language models

A method comprising:

• detecting that a parse error in a lexical text prevents generation of a graph that represents the lexical text;
• generating, by a token sequence encoder, an encoded sequence that represents the lexical text;
• and
• generating, based on the encoded sequence that represents the lexical text, an analytic result that reflects the lexical text, wherein the analytic result is selected from a group consisting of:
• a generation of a new name for the lexical text, a recognition of a portion of an identifier in the lexical text, a code completion of an expression in a logic statement in the lexical text, and a detection of semantic equivalence of the lexical text to a second lexical text.

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:

• generating a histogram that correctly represents a graph that represents a lexical text;
• generating a token sequence encoder that is trainable and untrained;
• inferring, by the token sequence encoder, an encoded sequence that incorrectly represents the lexical text;
• and
• adjusting the token sequence encoder based on a difference between the encoded sequence that incorrectly represents the lexical text and the histogram that correctly represents the graph.

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:

• detecting that a parse error in a lexical text prevents generation of a graph that represents the lexical text;
• generating, by a token sequence encoder, an encoded sequence that represents the lexical text;
• and
• generating, based on the encoded sequence that represents the lexical text, an analytic result that reflects the lexical text, wherein the analytic result is selected from a group consisting of:
• a generation of a new name for the lexical text, a recognition of a portion of an identifier in the lexical text, a code completion of an expression in a logic statement in the lexical text, and a detection of a semantic equivalence of the lexical text to a second lexical text.