Compact logging of network traffic events
AssigneeNetskope, Inc.
InventorRavindra K. Balupari
Life of this patent
Estimating the term…
Abstract
The technology disclosed works in real time, as base and subordinate HTTP URL requests are received, to attribute subordinate HTTP URL requests to base web pages. The main case uses the “referer” or “referrer” HTTP header field for attribution, directly and through a referer hierarchy to the base web page. A second case, which minimizes false generation of base web page log entries, involves small files, such as cascading style sheets (CSS) files, that often have a blank or no referer field. The technology disclosed applies equivalently to hypertext transfer protocol secure (HTTPS) data (e.g., HTTPS transactions, requests, and/or events).
Claims18
A computer-implemented method of compact logging of network traffic events, the method including:
- receiving a plurality of hypertext transfer protocol (abbreviated HTTP) requests produced during processing of a base page, including:
- a base unified resource locator (abbreviated URL),
- child URLs, and
- grandchild URLs;
- evaluating whether an HTTP request should be designated as the base URL, based on at least:
- a blank or no referer header field of the HTTP request,
- a content-length size of corresponding HTTP response above a predetermined threshold, and
- a text/HTML content-type of the HTTP response;
- identifying at least one HTTP request with a blank or no referer header field as referring to the base URL and initializing a first base page log object;
- identifying some of the HTTP requests as referred from the base URL, based on specification of the base URL in respective referer header fields, counting the identified HTTP requests as child URLs, and updating a first counter of the first base page log object;
- identifying some of the HTTP requests as referred from the child URLs, based on specification of one of the child URLs in respective referer header fields, counting the identified HTTP requests as grandchild URLs, and updating a second counter of the first base page log object;
- and
- identifying a further HTTP request with a blank or no referer header field as referring to a second base URL of a second base page, reporting out for use in a further process a summary of the first base page log object, and initializing a second base page log object.
A computer-implemented method of compact logging of network traffic events, the method including:
- receiving a plurality of subordinate unified resource locator (abbreviated URL) requests produced during loading a base web page;
- identifying subordinate URL requests as referred from the base web page, based on at least:
- a response content-length size below a predetermined threshold, and
- a response content-type other than text/HTML;
- and
- evaluating headers of the subordinate URL requests;
- identifying some of the subordinate URL requests as referred from the base web page, based on specification of a URL of the base web page in respective referer header fields of the subordinate URL requests;
- and
- handling logging of the identified subordinate URL requests as referred from the base web page by counting the identified subordinate URL requests and logging the count in a log entry for the base web page.
A system including one or more processors coupled to memory, the memory loaded with computer instructions to compactly log network traffic events, the instructions, when executed on the processors, implement actions comprising:
- receiving a plurality of hypertext transfer protocol (abbreviated HTTP) requests produced during processing of a base page, including
- a base unified resource locator (abbreviated URL),
- child URLs, and
- grandchild URLs;
- evaluating whether an HTTP request should be designated as the base URL, based on at least:
- a blank or no referer header field of the HTTP request,
- a content-length size of corresponding HTTP response above a predetermined threshold, and
- a text/HTML content-type of the HTTP response;
- identifying at least one HTTP request with a blank or no referer header field as referring to the base URL and initializing a first base page log object;
- identifying some of the HTTP requests as referred from the base URL, based on specification of the base URL in respective referer header fields, counting the identified HTTP requests as child URLs, and updating a first counter of the first base page log object;
- identifying some of the HTTP requests as referred from the child URLs, based on specification of one of the child URLs in respective referer header fields, counting the identified HTTP requests as grandchild URLs, and updating a second counter of the first base page log object;
- and
- identifying a further HTTP request with a blank or no referer header field as referring to a second base URL of a second base page, reporting out for use in a further process a summary of the first base page log object, and initializing a second base page log object.
Dependencies are inferred from the claim text.