System and method for the automatic recognition of relevant terms by mining link annotations

A computer program product for automatically and iteratively mining relevant terms comprising:

• a metadata extractor for extracting hypertext links from a document, the hypertext links containing metadata terms cn,m;
• a document vector module for creating a vector for the document, using the hypertext links;
• an association module for measuring the number of documents that contain the metadata terms cn,m in the hypertext links to perform a statistical analysis;
• wherein the association module discovers association rules from the document vector based primarily on the hypertext links;
• wherein the association rules comprise a support metric for an association rule (X|Y), where X and Y are sets of terms, and where a support p(X, Y) is defined as a joint probability of the frequency of co-occurrence of the sets of terms X and Y;
• and
• wherein the association rules further comprise a hybrid metric H(s,c) that normalize a support function n(s) and a confidence function n(c), and is expressed as follows:
• H(s,c)=a*n(s)+b*n(c),
• where s is the support metric, c is a confidence metric, and a and b are predefined weighting factors such that (a+b=1).

A method for automatically and iteratively mining relevant terms comprising:

• extracting hypertext links containing metadata terms cn,m from a document;
• creating a vector for the document, using the hypertext links;
• measuring the number of documents that contain the metadata terms cn,m in the hypertext links to perform a statistical analysis;
• discovering association rules from the document vector based primarily on the hypertext links;
• wherein the association rules comprise a support metric for an association rule (X|Y), where X and Y are sets of terms, and where a support p(X, Y) is defined as a joint probability of the frequency of co-occurrence of the sets of terms X and Y;
• and
• wherein the association rules further comprise a hybrid metric H(s,c) that normalize a support function n(s) and a confidence function n(c), and is expressed as follows:
• H(s,c)=a*n(s)+b*n(c),
• where s is the support metric, c is a confidence metric, and a and b are predefined weighting factors such that (a+b=1).

A computer program product having instruction codes for automatically and iteratively mining relevant terms comprising:

• a first set of instruction codes for extracting hypertext links from a document, the hypertext links containing metadata terms cn,m;
• a second set of instruction codes for creating a vector for the document, using the hypertext links;
• a third set of instruction codes for measuring the number of documents that contain the metadata terms cn,m in the hypertext links to perform a statistical analysis;
• wherein the third set of instruction codes discovers association rules from the document vector based primarily on the hypertext links;
• wherein the association rules comprise a support metric for an association rule (X|Y), where X and Y are sets of terms, and where a support p(X, Y) is defined as a joint probability of the frequency of co-occurrence of the sets of terms X and Y;
• and
• wherein the association rules further comprise a hybrid metric H(s,c) that normalize a support function n(s) and a confidence function n(c), and is expressed as follows:
• H(s,c)=a*n(s)+b*n(c),
• where s is the support metric, c is a confidence metric, and a and b are predefined weighting factors such that (a+b=1).

A system for automatically and iteratively mining relevant terms comprising:

• means for extracting hypertext links from a document, the hypertext links containing metadata terms cn,m;
• means for creating a vector for the document, using the hypertext links;
• means for measuring the number of documents that contain the metadata terms cn,m in the hypertext links to perform a statistical analysis;
• wherein the means for measuring the number of documents that contain the metadata terms cn,m in the hypertext, discovers association rules from the document vector based primarily on the hypertext links;
• wherein the association rules comprise a support metric for an association rule (X|Y), where X and Y are sets of terms, and where a support p(X, Y) is defined as a joint probability of the frequency of co-occurrence of the sets of terms X and Y;
• and
• wherein the association rules further comprise a hybrid metric H(s,c) that normalize a support function n(s) and a confidence function n(c), and is expressed as follows:
• H(s,c)=a*n(s)+b*n(c),
• where s is the support metric, c is a confidence metric, and a and b are predefined weighting factors such that (a+b=1).