# US 10,313,348 B2 · Document classification by a hybrid classifier > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,313,348 B2 - Document ID: US10313348B2 - Status: Granted - Kind: B2 - Application number: 15/268,977 - Filed: 2016-09-19 - Published: 2018-03-22 - Granted: 2019-06-04 - Earliest claimed: 2016-09-19 - Assignees: Fortinet, Inc. - Applicants: Fortinet, Inc. - Inventors: Ye Ma; Xiping Cao - Web page: https://patentgrep.com/patent/US10313348B2 ## CPC - H04L 63/10: {for controlling access to devices or network resources} - G06F 16/00: Information retrieval; Database structures therefor; File system structures therefor - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/00: Machine learning - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems and methods for classifying URLs by a hybrid classifier are provided. According to one embodiment, a hybrid classifier receives a candidate URL. A naïve Bayes classifier of the hybrid classifier classifies the candidate URL to generate a first classification of the candidate URL and a sublink classifier of the hybrid classifier classifies the candidate URL to generate a second classification of the candidate URL. The hybrid classifier determines a classification of the candidate URL based on the combination of the first and second classification. ## Claims ### Claim 1 A method comprising: receiving from a network security device protecting a private network, by a hybrid classification engine, a candidate uniform resource locator (URL) intercepted by the network security device as part of an attempt to access a network resource by a user of the private network and that is to be classified by the hybrid classification engine, wherein the hybrid classification engine comprises a naïve Bayes classifier and a sublink classifier; facilitating, by the hybrid classification engine, handling of the attempt to access the network resource by the network security device by performing a real-time classification of the candidate URL as being associated with one or more of a plurality of classes of content based on a combination of text content of the network resource and a plurality of sublinks contained within the network resource, including: generating a first classification for the candidate URL by classifying, by the naïve Bayes classifier, the candidate URL based on the text content and previous training of the naïve Bayes classifier on text content having a known association with the one or more of the plurality of classes of content; generating a second classification for the candidate URL by classifying, by the sublink classifier, the candidate URL, wherein the second classification is determined based on at least one sublink of the plurality of sublinks, by: retrieving the network resource referenced by the candidate URL; identifying the at least one sublink within the network resource; determining a classification of the at least one sublink by searching a URL classification database; and determining the second classification of the candidate URL based on the classification of the at least one sublink; and determining, by the hybrid classification engine, a final classification of the candidate URL by combining the first classification and the second classification, including: when the first classification is not within a plurality of restricted classes, determining whether a confidence score for the second classification is greater than a predetermined threshold indicative of a sufficient number of the plurality of sublinks being associated with a restricted class of the plurality of restricted classes; and responsive to determining the confidence score is greater than the predetermined threshold, setting the final classification to the restricted class, whereby the hybrid classification engine is able to classify the candidate URL as being associated with the restricted class despite an inability on a part of the naïve Bayes classifier to classify the candidate URL as being associated with the restricted class; and causing, by the hybrid classification engine, the network security device to block or allow the attempt to access the network resource based on the final classification. ### Claim 2 (depends on 1) The method of claim 1, further comprising determining, by the hybrid classification engine, the candidate URL is a restricted URL when either the first classification or the second classification is indicative of the candidate URL being associated with one of the plurality of restricted classes. ### Claim 3 (depends on 2) The method of claim 2, further comprising calculating, by the sublink classifier, the confidence score for the second classification based on the classification of the at least one sublink, wherein the confidence score is indicative of a probability that the candidate URL belongs to the second classification. ### Claim 4 (depends on 1) The method of claim 1, further comprising when the confidence score is less than the predetermined threshold, then concluding, by the hybrid classification engine, the candidate URL does not belong to the restricted class. ### Claim 5 (depends on 1) The method of claim 1, further comprising incorporating, by the hybrid classification engine, the classification within the URL classification database. ### Claim 6 (depends on 1) The method of claim 1, wherein the at least one sublink is an outlink that directs to a domain different from that of the candidate URL. ### Claim 7 A computer system comprising: non-transitory storage device having embodied therein instructions representing a hybrid classifier; and one or more processors coupled to the non-transitory storage device and operable to execute the hybrid classifier to perform a method comprising: receiving, from a network security device protecting a private network, a candidate uniform resource locator (URL) intercepted by the network security device as part of an attempt to access a network resource by a user of the private network and that is to be classified by the hybrid classifier, wherein the hybrid classifier implements a naïve Bayes classifier and a sublink classifier; facilitating handling of the access request by the network security device by performing a real-time classification of the candidate URL as being associated with one or more of a plurality of classes of content based on a combination of text content of the network resource and a plurality of sublinks contained within the network resource, including: generating a first classification for the candidate URL by classifying, by the naïve Bayes classifier, the candidate URL based on the text content and previous training of the naïve Bayes classifier on text content having a known association with the one or more of the plurality of classes of content; generating a second classification for the candidate URL by classifying, by the sublink classifier, the candidate URL, wherein the second classification is determined based on at least one sublink of the plurality of sublinks by: retrieving the network resource referenced by the candidate URL; identifying the at least one sublink within the network resource; determining a classification of the at least one sublink by searching a URL classification database; and determining the second classification of the candidate URL based on the classification of the at least one sublink; and determining a final classification of the candidate URL by combining the first classification and the second classification, including: when the first classification is not within a plurality of restricted classes, determining whether a confidence score for the second classification is greater than a predetermined threshold indicative of a sufficient number of the plurality of sublinks being associated with a restricted class of the plurality of restricted classes; and responsive to determining the confidence score is greater than the predetermined threshold, setting the final classification to the restricted class, whereby the hybrid classifier is able to classify the candidate URL as being associated with the restricted class despite an inability on a part of the naïve Bayes classifier to classify the candidate URL as being associated with the restricted class; and causing the network security device to block or allow the access request based on the final classification. ### Claim 8 (depends on 7) The computer system of claim 7, wherein the method further comprises determining the candidate URL is a restricted URL when either the first classification or the second classification is indicative of the candidate URL being associated with one of the plurality of restricted classes. ### Claim 9 (depends on 8) The computer system of claim 8, wherein the method further comprises calculating, by the sublink classifier, the confidence score for the second classification based on the classification of the at least one sublink, wherein the confidence score is indicative of a probability that the candidate URL belongs to the second classification. ### Claim 10 (depends on 7) The computer system of claim 7, wherein the method further comprises when the confidence score is less than the predetermined threshold, then concluding the candidate URL does not belong to the restricted class. ### Claim 11 (depends on 7) The computer system of claim 7, wherein the method further comprises incorporating the classification within the URL classification database. ### Claim 12 (depends on 7) The computer system of claim 7, wherein the at least one sublink is an outlink that directs to a domain different from that of the candidate URL. ### Claim 13 (depends on 1) The method of claim 1, wherein the hybrid classification engine is running on a local computer system within the private network. ### Claim 14 (depends on 1) The method of claim 1, wherein the hybrid classification engine is running on a computer system associated with a cloud-based security subscription service. ### Claim 15 (depends on 7) The computer system of claim 7, wherein the computer system is within the private network. ### Claim 16 (depends on 7) The computer system of claim 7, wherein the computer system is associated with a cloud-based security subscription service.