Systems and methods for categorizing web pages and using categorized web pages
AssigneePredict Interactive, Inc.
InventorTimothy Walton
Life of this patent
Estimating the term…
Abstract
Systems and methods for categorizing web pages and using categorized web pages are provided. The categorization involves using at least a master word list and one or more category word lists. The master word list tracks all words encountered by the system (except with the possible exception of natural words) and the category word lists includes words associated with a particular category. These lists can be initialized in a training phase and then the one or more category lists can be used to identify words in web pages matching words in one or more of the category lists. Based on this matching, as well as based on the master word list, a weight is generated for the web page, which is used to determine whether the web page belongs to a particular category.
Claims17
A method, comprising:
- receiving, by a processor, a master word list, a category word list, and a natural word list;
- receiving, by the processor, a web page;
- collecting, by the processor, human-readable words on the web page;
- for each of the collected human-readable words,
- discarding collected human-readable words matching words in the natural word list;
- adding each of the collected human-readable words that are not in the master word list to the master word list;
- and
- incrementing a count in the master word list for each of the collected human-readable words that are in the master word list;
- generating, by the processor, a weight for the received web page based on the collected human-readable words;
- determining, by the processor, whether the web page is assigned to a category of the category word list based on the generated weight;
- and
- outputting, by the processor, the web page as a categorized web page when it is determined the web page is assigned to a category of the category word list.
A method, comprising:
- receiving, by a processor, a master word list, a plurality of category word lists, and a natural word list;
- receiving, by the processor, a web page;
- collecting, by the processor, human-readable words on the web page;
- for each of the collected human-readable words,
- discarding collected human-readable words matching words in the natural word list;
- adding each of the collected human-readable words that are not in the master word list to the master word list;
- and
- incrementing a count in the master word list for each of the collected human-readable words that are in the master word list;
- generating, by the processor, a plurality of category weights for the received web page based on the collected human-readable words;
- determining, by the processor, whether the web page is assigned to a category of one of the plurality the category word list based on the generated category weights;
- and
- outputting, by the processor, the web page as a categorized web page when it is determined the web page is assigned to the category of one of the plurality of the category word lists.
A method, comprising:
- collecting, by a processor, human-readable words on a web page;
- generating, by the processor, a weight for the received web page by accumulating values associated with each of the human-readable words on the web page that match words in a category word list;
- determining, by the processor, that the web page is assigned to a category of the category word list based on the generated weight;
- and
- automatically generating human-readable content, by the processor, using the web page as a categorized web page when it is determined the web page is assigned to a category of the category word list;
- wherein the values associated with each of the human-readable words are based on a ratio of a frequency of occurrence in the category word list and a frequency of occurrence in a master word list.
Dependencies are inferred from the claim text.