Dynamic optimization of request parameters for proxy server

A method for data extraction and gathering from a web server comprising:

• receiving, by a request optimizer, a request from a user device;
• requesting, by the request optimizer, from a proxy rotator a proxy from a proxy database;
• accessing, by a proxy rotator, the proxy database;
• choosing, by the proxy rotator, from the proxy database the proxy;
• providing, by the proxy rotator, to the request optimizer information about the proxy;
• sending, by the request optimizer, the request from the user device to the proxy;
• obtaining, by the proxy, a reply from a target;
• forwarding, by the proxy, the reply obtained from the target to the request optimizer;
• analyzing, by the request optimizer, metadata of the reply;
• determining, by the request optimizer, pursuant to the analysis, if the reply is positive;
• and
• wherein, if the reply is positive, the reply is sent to the user device;
• wherein, if the reply is negative, the method for the data extraction and gathering from the web server is repeated, wherein the repeating comprises deciding, pursuant to the analysis of the reply, if an adjustment of the request is needed, and if the adjustment is needed, the request is modified to be a modified request, and the method for the data extraction and gathering from the web server repeats with the modified request in place of the request.