Building a complementary model for aggregating topics from textual content

A system, comprising:

• a processor;
• and
• a memory having instructions stored thereon which, when executed by the processor, performs an operation for generating a machine learning model for extracting information from textual content, the operation comprising:
• receiving a training data set including a plurality of documents having related textual strings;
• training, from the training data set, a relevancy model configured to generate relevance scores for a plurality of words extracted from the plurality of documents, wherein the relevance scores comprise scores generated for pairs of words in the plurality of documents based on a number of times a pair of words appears in the plurality of documents and an average distance between the pair of words;
• training, from the training data set, a knowledge graph model illustrating relationships between the plurality of words extracted from the plurality of documents, wherein training the knowledge graph model comprises truncating the knowledge graph model based on an entry degree of each node in the knowledge graph model, the truncating including removal of first nodes having an entry degree below a threshold value and connected to other nodes having an entry degree below the threshold value and removal of second nodes no longer connected to other nodes in the knowledge graph after removal of the first nodes;
• aggregating the relevancy model and the knowledge graph model into a complementary model, wherein aggregating the relevancy model and the knowledge graph model into the complementary model comprises, for each respective edge in the knowledge graph, adding a weight value based on a relevance score from the relevancy model between nodes connected by the respective edge in the knowledge graph;
• augmenting the complementary model to include additional nodes based on a text corpus received for analysis using the complementary model, each respective additional nodes being connected to an extant entity node in the knowledge graph and having an initial weight generated based on a relevancy score between the respective additional node and the extant entity node in the knowledge graph;
• deploying the complementary model;
• identifying, using the complementary model, a plurality of topics in an input document based on connections within a graph representing the input document and a threshold relevance score associated with the connections;
• and
• outputting, for display within a software application, the plurality of topics in conjunction with the input document.

A non-transitory computer-readable medium having instructions stored thereon which, when executed by a processor, performs an operation for generating a machine learning model for extracting information from textual content, the operation comprising:

• receiving a training data set including a plurality of documents having related textual strings;
• training, from the training data set, a relevancy model configured to generate relevance scores for a plurality of words extracted from the plurality of documents, wherein the relevance scores comprise scores generated for pairs of words in the plurality of documents based on a number of times a pair of words appears in the plurality of documents and an average distance between the pair of words;
• training, from the training data set, a knowledge graph model illustrating relationships between the plurality of words extracted from the plurality of documents, wherein training the knowledge graph model comprises truncating the knowledge graph model based on an entry degree of each node in the knowledge graph model, the truncating including removal of first nodes having an entry degree below a threshold value and connected to other nodes having an entry degree below the threshold value and removal of second nodes no longer connected to other nodes in the knowledge graph after removal of the first nodes;
• aggregating the relevancy model and the knowledge graph model into a complementary model, wherein aggregating the relevancy model and the knowledge graph model into the complementary model comprises, for each respective edge in the knowledge graph, adding a weight value based on a relevance score from the relevancy model between nodes connected by the respective edge in the knowledge graph;
• augmenting the complementary model to include additional nodes based on a text corpus received for analysis using the complementary model, each respective additional nodes being connected to an extant entity node in the knowledge graph and having an initial weight generated based on a relevancy score between the respective additional node and the extant entity node in the knowledge graph;
• deploying the complementary model;
• identifying, using the complementary model, a plurality of topics in an input document based on connections within a graph representing the input document and a threshold relevance score associated with the connections;
• and
• outputting, for display within a software application, the plurality of topics in conjunction with the input document.