System and method for url fetching retry mechanism

A method for fetching by a first device over the Internet a content that is stored only in a web server and is identified by a Uniform Resource Locator (URL), for use with a first and second groups of Internet Protocol (IP) addresses, the method comprising:

• first fetching, by the first device from the web server, a first response, the first fetching comprising:
• selecting, by the first device, a first IP address from the first group;
• sending, by the first device, a URL request to the web server using the selected first IP address;
• and
• receiving, by the first device, the first response;
• checking the first response for determining whether the first response is a proper response that comprises the content;
• responsive to determining that the first response is a proper response, using the fetched content;
• and
• responsive to determining that the first response is not a proper response, performing a second fetching, by the first device from the web server, of a second response, the second fetching comprising:
• selecting, by the first device, a second IP address from the second group;
• sending, by the first device, an additional URL request to the web server using the selected second IP address;
• receiving, by the first device, the second response;
• checking the second response for determining whether the second response is a proper response that comprises the content;
• responsive to determining that the first response is a proper response, using the fetched content;
• and
• responsive to determining that the second response is not a proper response, performing, by the first device, a first action.