Smart proxy rotator
AssigneeMetacluster Lt, Uab
InventorsMartynas Juravicius, Eivydas Vilcinskas
Life of this patent
Estimating the term…
Abstract
The task, logic of HTTP/HTTPS session statistics interception and collection is moved to the client side instead of the proxy layer. Encrypted HTTPS tunnel is terminated at the client end, making the actual content or data in transit invisible to both proxies and the smart proxy rotator (SPR). Client's scraping software has a plug-in installed that expands its functionality. HTTP/HTTPS session quality metrics are intercepted and collected at the client side, then sent to the SPR. Proxy usage mark “can be used” is obtained from the SPR for the currently analyzed proxy, based on the results of metrics analysis.
Claims15
A computer-implemented method for identifying a threshold of a proxy provider, the method comprising:
- running a proxy rotator on a computing device connected to a network;
- obtaining a proxy from a proxy provider by the proxy rotator;
- utilizing a web scraper to:
- establish a connection to a target server through the proxy;
- receive content designated within a request through the connection;
- provide performance and availability information of the proxy to the proxy rotator;
- utilizing the proxy rotator to calculate a weight for the proxy provider;
- and
- identifying a threshold of the proxy provider based on the calculated weight;
- wherein the proxy rotator stops using a maximized proxy provider, at least for a period of time, when at least one configured maximum threshold is reached;
- wherein the at least one configured maximum threshold is reached when a maximum number of concurrent connections is reached, a maximum error threshold is reached, or both the maximum number of concurrent connections is reached and the maximum error threshold is reached;
- wherein the content is unreadable to the proxy rotator due to network transport layer or application layer encryption.
A non-transitory computer-readable medium for identifying a threshold of a proxy provider, the medium comprising instructions that, when executed by a processor, direct the processor to:
- run a proxy rotator on a computing device connected to a network;
- obtain a proxy from a proxy provider by the proxy rotator;
- utilize a web scraper to:
- establish a connection to a target server through the proxy;
- receive content designated within a request through the connection;
- provide performance and availability information of the proxy to the proxy rotator;
- utilize the proxy rotator to calculate a weight for the proxy provider;
- identify a threshold of the proxy provider based on the calculated weight;
- and
- placing a usage of the proxy provider on hold, at least for a period of time, when at least one configured maximum threshold is reached;
- wherein the at least one configured maximum threshold is reached when a maximum number of concurrent connections is reached, a maximum error threshold is reached, or both the maximum number of concurrent connections is reached and the maximum error threshold is reached.
A computer-implemented method for identifying a threshold of a proxy provider, the method comprising:
- running a proxy rotator on a computing device connected to a network;
- obtaining a proxy from a proxy provider by the proxy rotator;
- utilizing a web scraper to:
- establish a connection to a target server through the proxy;
- receive content designated within a request through the connection;
- provide performance and availability information of the proxy to the proxy rotator;
- utilizing the proxy rotator to calculate a weight for the proxy provider;
- and
- identifying a threshold of the proxy provider based on the calculated weight;
- wherein the content is unreadable to the proxy rotator due to network transport layer or application layer encryption.
Dependencies are inferred from the claim text.