Categorization automation

A non-transitory computer readable storage medium storing instructions for categorization using a plurality of categories, the instructions comprising functionality to:

• obtain a plurality of uniform resource locators (URLs) associated with the plurality of categories;
• collect a plurality of web pages identified by the plurality of URLs;
• generate a plurality of vocabulary terms based on the plurality of web pages;
• generate an N-gram file including the plurality of vocabulary terms;
• generate a plurality of classified URLs by labeling the plurality of URLs based on the plurality of categories;
• generate a plurality of feature vectors by processing the classified URLs and the plurality of web pages against the N-gram file;
• generate a categorization model by applying a machine learning algorithm to the plurality of feature vectors;
• and
• load a classifier with the categorization module and the N-gram file,
• wherein the classifier is configured to select a category of the plurality of categories for an input associated with a user.

A system for categorization using a plurality of categories, comprising:

• a hardware processor;
• a data generation module configured to obtain a plurality of URLs associated with the plurality of categories and collect a plurality of web pages identified by the plurality of uniform resource locators;
• a vocabulary generator operatively connected to the data collector and configured to generate a plurality of vocabulary terms based on the plurality of web pages;
• an N-gram generator operatively connected to the vocabulary generator, and configured to generate an N-gram file including the plurality of vocabulary terms;
• a feature vector generator executing on the hardware processor, operatively connected to the N-gram generator, and configured to generate a plurality of feature vectors by processing the plurality of web pages and classified versions of the plurality of URLs against the N-gram file;
• a categorization model generator configured to generate a categorization model by applying a machine learning algorithm to the plurality of feature vectors;
• and
• a classifier loaded with the categorization model and the N-gram file and configured to predict a category of the plurality of categories based on an input associated with a user.