A system for searching, recommending, and exploring documents through conceptual associations

A system comprising:

• a memory having computer readable computer instructions;
• and
• a processor for executing the computer readable instructions, the computer readable instructions including:
• receiving a plurality of documents;
• extracting concepts and corresponding confidence scores from each of the documents, the confidence scores indicating for each extracted concept an estimate of confidence that the extracted concept is correct;
• computing a degree of relation between each of the documents and concepts in a concept graph;
• building a conceptual inverted table data structure based on the computing, the conceptual inverted table data structure indexed by concept and including, for each concept in the concept graph, a list of documents having associated degrees of relation to the concept greater than a threshold value and their associated degrees of relation to the concept;
• building a metadata structure that tracks a version number of each of the plurality of documents, wherein in response to the version number of a document being incremented, all computed degrees of relation between previous versions of the document and concepts in the concept graph are invalidated in the conceptual inverted table data structure, wherein prior to the invalidating, two versions of the document are listed in the conceptual inverted table data structure;
• and
• in response to receiving a query:
• determining one or more concepts from the query;
• for each of the determined concepts, accessing the conceptual inverted table data structure to retrieve a first list of documents having a highest degree of relation to the concept;
• and
• outputting a second list of documents responsive to the one or more first lists

A computer program product comprising:

• a tangible storage medium readable by a processing circuit and storing instructions for execution by the processing circuit to perform a method comprising:
• receiving a plurality of documents;
• extracting concepts and corresponding confidence scores from each of the documents, the confidence scores indicating for each extracted concept an estimate of confidence that the extracted concept is correct;
• computing a degree of relation between each of the documents and concepts in a concept graph;
• building a conceptual inverted table data structure based on the computing, the conceptual inverted table data structure indexed by concept and including, for each concept in the concept graph, a list of documents having associated degrees of relation to the concept greater than a threshold value and their associated degrees of relation to the concept;
• building a metadata structure that tracks a version number of each of the plurality of documents, wherein in response to the version number of a document being incremented, all computed degrees of relation between previous versions of the document and concepts in the concept graph are invalidated in the conceptual inverted table data structure, wherein prior to the invalidating, two versions of the document are listed in the conceptual inverted table data structure;
• and
• in response to receiving a query:
• determining one or more concepts from the query;
• for each of the determined concepts, accessing the conceptual inverted table data structure to retrieve a first list of documents having a highest degree of relation to the concept;
• and
• outputting a second list of documents responsive to the one or more first lists