Back to results
US 12,762,209 B2Granted
Insufficient content detection with machine learning ensemble
AssigneePalo Alto Networks, Inc.
InventorsZeyu You, Wei Wang, Jingwei Fan
Life of this patent
Estimating the term…
Abstract
An insufficient content (IC) detection ensemble comprising a natural language model and a gradient boosting classifier detects IC in HyperText Transfer Protocol (HTTP) responses corresponding to Uniform Resource Locators (URLs). The architecture of the IC detection ensemble is such that the natural language model receives natural language tokens from body elements of HTML code in the HTTP responses as inputs, and the gradient boosting classifier receives count-based feature values and additional feature values extracted from the HTTP responses and outputs from the natural language model to generate IC/non-IC verdicts.
Claims20
A method comprising:
- extracting content from one or more HyperText Transfer Protocol (HTTP) responses based on HTTP requests to a first Uniform Resource Locator (URL);
- inputting the content into a natural language model to generate first one or more likelihood values that a webpage corresponding to the first URL comprises insufficient content;
- generating a plurality of feature values based, at least in part, on the one or more HTTP responses, wherein the plurality of feature values comprises first values of one or more count-based features in the content and second values of one or more additional features, wherein the second values of the one or more additional features comprise at least one of values of fields extracted from the one or more HTTP responses and indications of one or more login forms in the one or more HTTP responses;
- inputting the plurality of feature values and the first one or more likelihood values into a first classifier to obtain a second likelihood value that the webpage corresponding to the first URL comprises insufficient content as output, wherein the first classifier was trained as an ensemble with the natural language model to output augment predictions of the natural language model with feature values of the one or more count-based features and the one or more additional features when predicting whether URLs correspond to insufficient content;
- and
- indicating the first URL as corresponding to insufficient content based, at least in part, on the second likelihood value.
3 more elements
A non-transitory machine-readable medium having program code stored thereon, the program code comprising instructions to:
- crawl a first plurality of Uniform Resource Locators (URLs) to obtain a plurality of HyperText Transfer Protocol (HTTP) responses;
- parse each of the plurality of HTTP responses to generate plurality of feature values and natural language content, wherein the plurality of feature values comprises first values of one or more count-based features in the natural language content and second values of one or more additional features, wherein the second values of the one or more additional features comprise at least one of values of fields extracted from the plurality of HTTP responses and indications of one or more login forms in the plurality of HTTP responses;
- and
- train an ensemble of a natural language model and a classifier to predict whether each of the first plurality of URLs corresponds to incomplete content, wherein the natural language model takes the natural language content as input and the classifier takes outputs of the natural language model and the plurality of feature values as inputs, wherein the natural language model is trained as an ensemble with the classifier to augment predictions by the natural language model using feature values of the one or more count-based features and the one or more additional features.
1 more element
An apparatus comprising:
- a processor;
- and
- a machine-readable medium having instructions stored thereon that are executable by the processor to cause the apparatus to,
- communicate a HyperText Transfer Protocol (HTTP) request for a first Uniform Resource Locator (URL);
- generate a plurality of feature values from an HTTP response returned responsive to the HTTP request, wherein the plurality of feature values comprises first values of one or more count-based features of content in the HTTP response and second values of one or more additional features, wherein the second values of the one or more additional features comprise at least one of values of fields extracted from the HTTP response and indications of one or more login forms in the HTTP response;
- input the plurality of feature values into an ensemble of a natural language model and a first classifier, wherein first inputs to the natural language model comprise a first subset of the plurality of feature values, wherein second inputs to the first classifier comprise outputs of the natural language model from inputting the first inputs and a second subset of the plurality of feature values, wherein the first classifier was trained as an ensemble with the natural language model to augment predictions of the natural language model with feature values of the one or more count-based features and the one or more additional features when predicting whether URLs correspond to insufficient content;
- and
- indicate the first URL as corresponding to non-categorizable content based, at least in part, on one or more outputs of the ensemble from inputting the plurality of feature values.
5 more elements
Dependencies are inferred from the claim text.