System and method for dynamically evaluating latent concepts in unstructured documents

A computer-implemented method for analyzing unstructured documents for conceptual relationships, comprising:

• determining a frequency of occurrences of concepts in a set of unstructured documents, each concept representing an element occurring in one or more of the unstructured documents;
• selecting a subset of concepts out of the frequency of occurrences;
• grouping one or more concepts from the concepts subset;
• assigning weights to one or more clusters of concepts for each group of concepts;
• and
• calculating a best fit approximation for each document indexed by each such group of concepts between the frequency of occurrences and the weighted cluster for each such concept grouped into the group of concepts.

A computer-implemented system for dynamically evaluating latent concepts in unstructured documents, comprising:

• an extraction module extracting a multiplicity of concepts from a set of unstructured documents into a lexicon uniquely identifying each concept and a frequency of occurrence;
• a frequency mapping module creating a frequency of occurrence representation for each documents set, the representation providing an ordered corpus of the frequencies of occurrence of each concept;
• a concept selection module selecting a subset of concepts from the frequency of occurrence representation filtered against a minimal set of concepts each referenced in at least two documents with no document in the corpus being unreferenced;
• a group generation module generating a group of weighted clusters of concepts selected from the concepts subset;
• and
• a best fit module determining a matrix of best fit approximations for each document weighted against each group of weighted clusters of concepts.

A computer-implemented method for dynamically evaluating latent concepts in unstructured documents, comprising:

• extracting a multiplicity of concepts from a set of unstructured documents into a lexicon uniquely identifying each concept and a frequency of occurrence;
• creating a frequency of occurrence representation for each documents set, the representation providing an ordered corpus of the frequencies of occurrence of each concept;
• selecting a subset of concepts from the frequency of occurrence representation filtered against a minimal set of concepts each referenced in at least two documents with no document in the corpus being unreferenced;
• generating a group of weighted clusters of concepts selected from the concepts subset;
• and
• determining a matrix of best fit approximations for each document weighted against each group of weighted clusters of concepts.