Automatic genre classification determination of web content to which the web content belongs together with a corresponding genre probability

A system comprising:

• a hardware processor;
• and
• a memory coupled to the hardware processor, wherein the memory comprises instructions which, when executed by the hardware processor, cause the hardware processor to:
• for each of at least one type of web content genre to be trained in a training process:
• extract a set of feature types comprising genre features and non-genre features from the collected first type of training material and second type of training material respectively, wherein the genre features and the non-genre features are represented by tokens consisting of fixed length character strings extracted from content strings of the first and second type of training material;
• and
• store each token in a corresponding feature database together with a first integer count representing a frequency of appearance of the token in the first type of training material and a second integer count representing a frequency of appearance of the token in the second type of training material;
• and
• in a classification process:
• provide web content, wherein a first feature type is generated by joining together attribute values of all HTML meta data tags to form a single content string, wherein each attribute value is separated by a single space character, further text content from a HTML title tag and HTML anchor tags is extracted and appended to the content string, wherein characters are converted to lower case and only alpha-numeric and space characters are added to the content string and sequences of space characters are compressed to a single space character;
• wherein a second feature type is generated by joining together attribute values of all HTML anchor tags and all link tags to form a single content string, wherein each attribute value is separated by a single space character, and wherein characters are converted to lower case and only alpha-numeric and space characters are added to the content string and sequences of space characters are compressed to a single space character;
• extract fixed length tokens for each feature type of the set of feature types from different text and structural elements of the web content;
• calculate for each feature type of the set of feature types a corresponding feature probability that the web content belongs to a corresponding specific trained web content genre by combining probabilities of the genre features and non-genre features;
• combine the feature probabilities to an overall genre probability that the web content belongs to a specific trained web content genre;
• and
• output a genre classification result comprising at least one specific trained web content genre to which the web content belongs together with a corresponding genre probability.

A computer program product comprising a non-transitory computer readable storage medium having a computer-readable program stored therein, wherein the computer-readable program, when executed on a computing device, causes the computing device to:

• for each of at least one type of web content genre to be trained in a training process:
• extract a set of feature types comprising genre features and non-genre features from the collected first type of training material and second type of training material respectively, wherein the genre features and the non-genre features are represented by tokens consisting of fixed length character strings extracted from content strings of the first and second type of training material;
• and
• store each token in a corresponding feature database together with a first integer count representing a frequency of appearance of the token in the first type of training material and a second integer count representing a frequency of appearance of the token in the second type of training material;
• and
• in a classification process:
• provide web content, wherein a first feature type is generated by joining together attribute values of all HTML meta data tags to form a single content string, wherein each attribute value is separated by a single space character, further text content from a HTML title tag and HTML anchor tags is extracted and appended to the content string, wherein characters are converted to lower case and only alpha-numeric and space characters are added to the content string and sequences of space characters are compressed to a single space character;
• wherein a second feature type is generated by joining together attribute values of all HTML anchor tags and all link tags to form a single content string, wherein each attribute value is separated by a single space character, and wherein characters are converted to lower case and only alpha-numeric and space characters are added to the content string and sequences of space characters are compressed to a single space character;
• extract fixed length tokens for each feature type of the set of feature types from different text and structural elements of the web content;
• calculate for each feature type of the set of feature types a corresponding feature probability that the web content belongs to a corresponding specific trained web content genre by combining probabilities of the genre features and non-genre features;
• combine the feature probabilities to an overall genre probability that the web content belongs to a specific trained web content genre;
• and
• output a genre classification result comprising at least one specific trained web content genre to which the web content belongs together with a corresponding genre probability.