Method and system for assessing similarity of documents

A method for assessing similarity of documents, comprising:

• extracting a reference document text from a reference document;
• extracting an archived document text from an archived document;
• quantifying the reference document, comprising:
• tokenizing reference document sentences of the reference document text;
• and
• vectorizing the tokenized reference document sentences to obtain a plurality of reference document paths;
• quantifying the archived document, comprising:
• tokenizing archived document sentences of the archived document text;
• and
• vectorizing the tokenized archived document sentences to obtain a plurality of archived document paths;
• determining a document similarity value of the quantified reference document and the quantified archived document, comprising:
• selecting a first reference document path from the plurality of reference document paths;
• selecting a first archived document path from the plurality of archived document paths;
• making a first determination that a length of the first reference document path is different from a length of the first archived document path;
• based on the first determination:
• identifying a shorter path and a longer path of the first reference document path and the first archived document path;
• generating subsequences from the longer path, with a length matching the length of the shorter path;
• calculating a path similarity value for the shorter path and the subsequences;
• adding the path similarity value to a sum, wherein the sum is the document similarity value.

A non-transitory computer readable medium (CRM) storing instructions for assessing similarity of documents, the instructions comprising functionality for:

• extracting a reference document text from a reference document;
• extracting an archived document text from an archived document;
• quantifying the reference document, comprising:
• tokenizing sentences of the reference document;
• and
• vectorizing the tokenized sentences to obtain a reference document text vector for each sentence of the reference document;
• quantifying the archived document, comprising:
• tokenizing sentences of the archived document;
• and
• vectorizing the tokenized sentences to obtain an archived document text vector for each sentence of the archived document;
• determining a document similarity value of the quantified reference document and the quantified archived document, comprising:
• calculating a plurality of vector similarity values for a plurality of combinations of a reference document text vector and an archived document text vector, the calculating including multiplying each of the vector similarity values of the plurality of vector similarity values with a corresponding weight that is determined, based at least in part on a number of skipped tokens;
• and
• calculating the document similarity value, comprising a sum of the plurality of vector similarity values.

A non-transitory computer readable medium (CRM) storing instructions for assessing similarity of documents, the instructions comprising functionality for:

• extracting a reference document text and reference document metadata from a reference document;
• extracting an archived document text and archived document metadata from an archived document;
• quantifying the reference document, comprising:
• tokenizing sentences of the reference document;
• and
• vectorizing the tokenized sentences to obtain a reference document text vector for each sentence of the reference document;
• quantifying the archived document, comprising:
• tokenizing sentences of the archived document;
• and
• vectorizing the tokenized sentences to obtain an archived document text vector for each sentence of the archived document;
• determining a document similarity value of the quantified reference document and the quantified archived document, comprising:
• calculating a first plurality of vector similarity values for a plurality of combinations of a reference document text vector and an archived document text vector;
• calculating a second plurality of vector similarity values based on combinations of reference document metadata vectors that are based on the reference document metadata and archived document metadata vectors that are based on the archived document metadata;
• and
• calculating the document similarity value, comprising a sum of the first plurality of vector similarity values and a sum of the second plurality of vector similarity values.

A method for assessing similarity of documents, comprising:

• extracting a reference document text from a reference document;
• extracting an archived document text from an archived document;
• quantifying the reference document, comprising:
• tokenizing sentences of the reference document;
• and
• vectorizing the tokenized sentences to obtain a reference document text vector for each sentence of the reference document;
• quantifying the archived document, comprising:
• tokenizing sentences of the archived document;
• and
• vectorizing the tokenized sentences to obtain an archived document text vector for each sentence of the archived document;
• determining a document similarity value of the quantified reference document and the quantified archived document, comprising:
• calculating a plurality of vector similarity values for a plurality of combinations of a reference document text vector and an archived document text vector, the calculating including multiplying each of the vector similarity values of the plurality of vector similarity values with a corresponding weight that is determined, based at least in part on a number of skipped tokens;
• and
• calculating the document similarity value, comprising a sum of the plurality of vector similarity values.