Standard exact clause detection

A computer implemented method for determining a presence of a type of clause within a plurality of documents, the method comprising:

• receiving a clause example corresponding to the type of clause;
• generating a primary policy based upon the received clause example for use in a semantic language evaluator configured to assess a level of semantic similarity between received clauses, the primary policy comprising one or more policy rules and associated with a first threshold value-indicating a level of semantic similarity of a clause to the clause example;
• analyzing, using the semantic language evaluator, the plurality of documents according to the primary policy to automatically provide a first set of clauses corresponding of the plurality of documents, each clause of the first set corresponding to a standard clause matching the clause example in accordance with the first threshold;
• generating a mirror document based upon the plurality of documents by automatically replacing one or more portions of the plurality of documents having allowable variations with corresponding variables;
• parsing the mirror document to generate a second set of clauses corresponding to a standard exact feature data set;
• generating a secondary policy based upon the primary policy and the clause example for use in the semantic language evaluator, the secondary policy associated with a second threshold value indicating a level of semantic similarity of a clause to the clause example that is lower than the first threshold value;
• analyzing, using the semantic language evaluator, the plurality of documents according to the secondary policy to automatically provide a third set of clauses comprising non-standard clauses semantically related to but not matching the clause example in accordance with the second threshold, wherein the third set of clauses corresponds to a mirror feature data set;
• obtaining a difference between the mirror feature data set and the standard exact feature data set, the difference corresponding to non-standard clauses of the plurality of documents;
• and
• automatically updating a database to identify the standard and non-standard clauses of the plurality of documents associated with the type of clause based upon the obtained difference, for subsequent usage in analyzing the plurality of documents.

A system for determining a presence of a type of clause within a plurality of documents, comprising:

• a document parsing module configured to receive a clause example corresponding to the type of clause;
• a policy definition module configured to:
• generate a primary policy based upon the received clause example for use in a semantic language evaluator configured to assess a level of semantic similarity between received clauses, the primary policy comprising one or more policy rules and associated with a first threshold value indicating a level of semantic similarity of a clause to the clause example;
• and
• generate a secondary policy based upon the primary policy and the clause example for use in the semantic language evaluator, the secondary policy associated with a second threshold value indicating a level of semantic similarity of a clause to the clause example that is lower than the first threshold value;
• an analysis engine configured to:
• analyze, using the semantic language evaluator, the plurality of documents according to the primary policy to automatically provide a first set of clauses corresponding of the plurality of documents, each clause of the first set corresponding to a standard clause matching the clause example in accordance with the first threshold;
• generate a mirror document based upon the plurality of documents by automatically replacing one or more portions of the plurality of documents having allowable variations with corresponding variables;
• parse the mirror document to generate a second set of clauses corresponding to a standard exact feature data set;
• analyze, using the semantic language evaluator, the plurality of documents according to the secondary policy to automatically provide a third set of clauses comprising non-standard clauses semantically related to but not matching the clause example in accordance with the second threshold, wherein the third set of clauses corresponds to a minor feature data set;
• obtain a difference between the mirror feature data set and the standard exact feature data set, the difference corresponding to non-standard clauses of the plurality of documents;
• and
• update, automatically, a database to identify the standard and non-standard clauses of the plurality of documents associated with the type of clause based upon the obtained difference, for subsequent usage in analyzing the plurality of documents.

A non-transitory computer readable medium comprising stored program code for determining a presence of a type of clause within a plurality of documents, the program code comprising instructions that when executed by a processor cause the processor to:

• receive a clause example corresponding to the type of clause;
• generate a primary policy based upon the received clause example for use in a semantic language evaluator configured to assess a level of semantic similarity between received clauses, the primary policy comprising one or more policy rules and associated with a first threshold value indicating a level of semantic similarity of a clause to the clause example;
• analyze, using the semantic language evaluator, the plurality of documents according to the primary policy to automatically provide a first set of clauses of the plurality of documents, each clause of the first set corresponding to a standard clause matching the clause example in accordance with the first threshold;
• generate a mirror document based upon the plurality of documents by automatically replacing one or more portions of the plurality of documents having allowable variations with corresponding variables;
• generate a secondary policy based upon the primary policy and the clause example for use in the semantic language evaluator, the secondary policy associated with a second threshold value indicating a level of semantic similarity of a clause to the clause example that is lower than the first threshold value;
• analyze, using the semantic language evaluator, the mirror document according to the secondary policy to automatically provide a second set of clauses corresponding to a mirror feature data set comprising non-standard clauses semantically related to but not matching the clause example in accordance with the second threshold;
• obtain a difference between the mirror feature data set and the first set of clauses, the difference corresponding to non-standard clauses of the plurality of documents;
• and
• update, automatically, a database to identify the standard and non-standard clauses of the plurality of documents associated with the type of clause based upon the obtained difference, for subsequent usage in analyzing the plurality of documents.