Measuring accuracy of semantic graphs with exogenous datasets

A system, comprising:

• one or more processors;
• memory storing instructions that when executed by at least some of the processors effectuate operations comprising:
• obtaining a weighted semantic graph of semantic similarity between unstructured text in documents within an analyzed corpus, wherein weights of the semantic graph are inferred by unsupervised learning of the weights by one or more computers, and wherein the semantic graph comprises:
• more than 1000 nodes, each corresponding to at least one respective document within the analyzed corpus;
• and
• more than 2000 weighted edges, each weighted edge linking two of the nodes and having a score indicating an amount of semantic similarity between documents corresponding to the two linked nodes;
• obtaining access to an external corpus having at least some other documents with unstructured text about entities mentioned in the analyzed corpus, the other documents not being within the analyzed corpus;
• for each of at least 20 evaluation nodes among the nodes of the graph, by one or more processors, scoring semantic similarity between documents in the analyzed corpus and documents in the external corpus selected as being associated with adjacent nodes to the respective evaluation node, wherein scoring semantic similarity comprises:
• determining the adjacent node in the graph based on the adjacent node sharing an edge with the respective evaluation node;
• selecting one or more documents from the external corpus based on the selected documents being associated with the adjacent node;
• determining n-gram weights of a plurality of n-grams in text of the document corresponding to the adjacent node based on the weight of the edge linking the respective evaluation node to the adjacent node in the semantic graph;
• and
• determining one or more exogenous semantic similarity scores between the documents selected from the external corpus and the respective evaluation node, the exogenous semantic similarity scores being determined based on the determined n-gram weights and the presence of corresponding n-grams in the respective documents selected from the external corpus;
• and
• determining, by one or more processors, a measure of quality of at least some of the weighted edges of the semantic graph based on the exogenous semantic similarity scores.