# US 2025/0061163 A1 · System and method for url fetching retry mechanism

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0061163 A1
- Document ID: US20250061163A1
- Status: Published application
- Kind: A1
- Application number: 18/938,852
- Filed: 2024-11-06
- Published: 2025-02-20
- Earliest claimed: 2019-02-25
- Assignees: Bright Data Ltd.
- Applicants: Bright Data Ltd.
- Inventors: Derry Shribman; Ofer Vilenski
- Web page: https://patentgrep.com/patent/US20250061163A1

## CPC

- G06F 16/955: using information identifiers, e.g. uniform resource locators [URL]
- G06F 9/547: {Remote procedure calls [RPC]; Web services}
- G06F 16/909: using geographical or spatial information, e.g. location (spatiotemporally dependent retrieval from the web G06F16/9537)
- G06F 16/9558: {Details of hyperlinks; Management of linked annotations}
- G06F 16/9566: {URL specific, e.g. using aliases, detecting broken or misspelled links}
- H04L 1/00: Arrangements for detecting or preventing errors in the information received {(correcting synchronisation H04L7/00)}
- H04L 63/1466: {Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks}
- H04L 67/01: Protocols
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/1014: based on the content of a request
- H04L 67/1021: based on client or server locations
- H04L 67/1036: Load balancing of requests to servers for services different from user content provisioning, e.g. load balancing across domain name servers
- H04L 67/146: Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
- H04L 67/563: Data redirection of data network streams
- H04L 67/564: Enhancement of application control based on intercepted application data
- H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching
- H04L 69/40: for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection (management of faults, events, alarms or notifications in data switching networks H04L41/06)
- H04L 61/59: using proxies for addressing
- H04L 63/0272: {Virtual private networks}
- H04L 63/145: {the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms}
- H04L 63/1483: {service impersonation, e.g. phishing, pharming or web spoofing (detection of rogue wireless access points H04W12/12)}
- H04L 63/168: {above the transport layer}
- H04L 67/56: Provisioning of proxy services (store-and-forward switching systems in data switching networks H04L12/54)
- H04L 67/63: Routing a service request depending on the request content or context
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A method for overcoming intermittent, temporary, or other fetching failures by using multiple attempts for retrieving a content from a web server to a client device is disclosed. The URL fetching may use direct or non-direct fetching schemes, or a combination thereof. The non-direct fetching method may use intermediate devices, such as proxy server, Data-Center proxy server, tunnel devices, or any combination thereof. Upon sensing a failure of a fetching action, the action is repeated using the same or different parameters or attributes, such as by using different intermediate devices, selected based on different parameters or attributes, such as different countries. The repetitions are limited to a pre-defined maximum number or attempts. The fetching attempts may be performed by the client device, by an intermediate device in a non-direct fetching scheme, or a combination thereof. Various fetching schemes may be used sequentially until the content is retrieved.

## Claims

### Claim 1

A method for use with a Uniform Resource Locator (URL) and for use with a group of devices where each of the devices in the group is not a web server, the method comprising:
selecting, by a requesting device, a first device from the group;
sending, by the requesting device over the Internet to the selected first device, a first Hypertext Transfer Protocol (HTTP) that comprises the URL;
receiving, by the requesting device from the first device over the Internet, in response to the sending of the URL to the first device, a first content that is identified by the URL;
selecting, by the requesting device, a second device from the group;
sending, by the requesting device over the Internet to the selected second device, a second Hypertext Transfer Protocol (HTTP) that comprises the URL;
receiving, by the requesting device from the second device over the Internet, in response to the sending of the URL to the second device, a second content that is identified by the URL;
selecting, by the requesting device, a third device from the group;
sending, by the requesting device over the Internet to the selected third device, a third Hypertext Transfer Protocol (HTTP) that comprises the URL;
receiving, by the requesting device from the third device over the Internet, in response to the sending of the URL to the third device, a third content that is identified by the URL;
comparing, by the requesting device, the received first, second, and third contents; and
using, by the requesting device, the received first, second, and third contents.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the using comprises displaying to a user, using a web browser in the requesting device, in response to the received first, second, and third contents.

### Claim 3 (depends on 1)

The method according to claim 1, wherein each of the first, second, or third content comprises a respective web-page or a respective part of a respective web-page.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the using comprises identifying differences between the received first, second, and third contents.

### Claim 5 (depends on 1)

The method according to claim 1, for use with a web server that stores the first content, the method further comprising:
receiving, by the first device from the requesting device over the Internet, the URL;
sending, by the first device to the web server over the Internet, the URL, in response to the receiving of the URL, using an Internet Protocol (IP) address of the first device as source address;
receiving, by the first device from the web server over the Internet, the first content, in response to the sending of the URL; and
sensing, by the first device to the requesting device over the Internet, the received first content.

### Claim 6 (depends on 1)

The method according to claim 1, for use with a web server that stores the second content, the method further comprising:
receiving, by the second device from the requesting device over the Internet, the URL;
sending, by the second device to the web server over the Internet, the URL, in response to the receiving of the URL, using an Internet Protocol (IP) address of the second device as source address;
receiving, by the second device from the web server over the Internet, the second content, in response to the sending of the URL; and
sensing, by the second device to the requesting device over the Internet, the received second content.

### Claim 7 (depends on 1)

The method according to claim 1, for use with a web server that stores the third content, the method further comprising:
receiving, by the third device from the requesting device over the Internet, the URL;
sending, by the third device to the web server over the Internet, the URL, in response to the receiving of the URL, using an Internet Protocol (IP) address of the third device as source address;
receiving, by the third device from the web server over the Internet, the third content, in response to the sending of the URL; and
sensing, by the third device to the requesting device over the Internet, the received third content.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising receiving, by the requesting device from a client device, the URL.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the requesting device comprises a server device.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the server device consists of, includes, is part of, or is integrated with, a proxy server.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the proxy server comprises a Data-Center (DC) proxy-server.

### Claim 12 (depends on 10)

The method according to claim 10, wherein the proxy server consists of, includes, is part of, or is integrated with, an HTTP proxy server, a web-proxy server, a caching proxy, an open-source caching proxy server, a cloud-based proxy server, an open proxy server, a forwarding proxy server, a reverse proxy server, a transparent proxy server, a non-transparent proxy server, an anonymous proxy server, a translation proxy server, a SOCKS proxy server, a Common Gateway Interface (CGI) web proxy server, a suffix proxy server, an I2P anonymous proxy server, a Domain Name System (DNS) proxy server, or any combination thereof.

### Claim 13 (depends on 8)

The method according to claim 8, further comprising sending the first, second, and third contents or a part thereof, to the client device.

### Claim 14 (depends on 1)

The method according to claim 1, wherein the first content comprises, or consists of, a web-page that includes, consists of, or comprises, a part of, or whole of, a data file, a text data, audio data, voice data, multimedia data, video data, an image, music data, a computer program, or any combination thereof.

### Claim 15 (depends on 1)

The method according to claim 1, further for use with a virtualization, wherein the first, second, or third device consists of, comprises, is part of, or is integrated with, a server device that virtualizes a client device.

### Claim 16 (depends on 15)

The method according to claim 15, wherein the client device virtualization executed as part of a Virtual Machine (VM).

### Claim 17 (depends on 16)

The method according to claim 16, for use with a host computer that implements the VM, wherein the method further comprising executing, by the host computer, a hypervisor or a Virtual Machine Monitor (VMM).

### Claim 18 (depends on 15)

The method according to claim 15, wherein the virtualization includes, is based on, or uses, full virtualization, para-virtualization, or hardware assisted virtualization.

### Claim 19 (depends on 1)

The method according to claim 1, wherein at least one of the first, second, and third devices comprises a client device.

### Claim 20 (depends on 19)

The method according to claim 19, wherein each one of the first, second, and third devices comprises a respective client device.

### Claim 21 (depends on 1)

The method according to claim 1, wherein the first Hypertext Transfer Protocol (HTTP) request comprises a Hypertext Transfer Protocol Secure (HTTPS) request.

### Claim 22 (depends on 1)

The method according to claim 1, wherein the second Hypertext Transfer Protocol (HTTP) request comprises a Hypertext Transfer Protocol Secure (HTTPS) request.

### Claim 23 (depends on 1)

The method according to claim 1, wherein the third Hypertext Transfer Protocol (HTTP) request comprises a Hypertext Transfer Protocol Secure (HTTPS) request.

### Claim 24 (depends on 1)

The method according to claim 1, wherein the first, second, and third contents are stored in a web server.

### Claim 25 (depends on 1)

The method according to claim 1, further comprising storing, by the requesting device, the received first content.

### Claim 26 (depends on 1)

The method according to claim 1, further comprising storing, by the requesting device, the received second content.

### Claim 27 (depends on 1)

The method according to claim 1, further comprising storing, by the requesting device, the received third content.

### Claim 28 (depends on 1)

The method according to claim 1, further comprising storing, by the requesting device, the received first, second, and third content.

### Claim 29 (depends on 1)

The method according to claim 1, further for use with a list of Internet Protocol (IP) addresses, wherein each of the IP addresses in the list is associated with a distinct device in the group, and wherein the selecting comprises selecting the respective IP address from the list.

### Claim 30 (depends on 1)

The method according to claim 1, wherein the selecting of the first, second, or third device is based on, or uses, load balancing.

### Claim 31 (depends on 1)

The method according to claim 1, wherein the selecting of the first, second, or third device is based on, or uses, random selection.

### Claim 32 (depends on 31)

The method according to claim 31, wherein random selection uses, or is based on, one or more random numbers generated by a random number generator.

### Claim 33 (depends on 31)

The method according to claim 31, wherein the random number generator is hardware based.

### Claim 34 (depends on 33)

The method according to claim 33, wherein the random number generator is using thermal noise, shot noise, nuclear decaying radiation, photoelectric effect, or quantum phenomena.

### Claim 35 (depends on 31)

The method according to claim 31, wherein the random number generator is software based.

### Claim 36 (depends on 35)

The method according to claim 35, wherein the random number generator is based on executing an algorithm for generating pseudo-random numbers.

### Claim 37 (depends on 1)

The method according to claim 1, wherein the selecting of the first, second, or third device is based on, or uses, an estimated geographical location of the requesting device or of a web server.

### Claim 38 (depends on 37)

The method according to claim 37, further comprising estimating the geographical location of the requesting device or of the web server using geolocation.

### Claim 39 (depends on 38)

The method according to claim 38, wherein the geolocation is based on IP geolocation.

### Claim 40 (depends on 38)

The method according to claim 38, wherein the geolocation is based on W3C Geolocation Application Programming Interface (API).

### Claim 41 (depends on 37)

The method according to claim 37, wherein the first, second, or third device is selected based on estimated as being in the same area as the requesting device or the web server.

### Claim 42 (depends on 41)

The method according to claim 41, wherein the first, second, or third device is selected based on being estimated as being in the same continent, country, state, region, city, postal/zip code, latitude, longitude, or Timezone as the requesting device or the web server.

### Claim 43 (depends on 1)

The method according to claim 1, wherein the first, second, or third device is selected based on being a recent one to be selected, or based on being a least recent to be selected.

### Claim 44 (depends on 1)

The method according to claim 1, wherein the first, second, or third device is manually selected by a user.

### Claim 45 (depends on 1)

The method according to claim 1, wherein the requesting device, the first device, the second device, or the third device, consists or, comprises, or is part of, a client device in a client/server architecture.

### Claim 46 (depends on 45)

The method according to claim 45, wherein the client device is housed in a single enclosure that is a hand-held enclosure or a portable enclosure.

### Claim 47 (depends on 45)

The method according to claim 45, wherein the client device consists of, comprises, is part of, or is integrated with, a notebook computer, a laptop computer, a media player, a Digital Still Camera (DSC), a Digital video Camera (DVC or digital camcorder), a Personal Digital Assistant (PDA), a cellular telephone, a digital camera, a video recorder, a smartphone, or any combination thereof.

### Claim 48 (depends on 47)

The method according to claim 47, wherein the client device consists of, comprises, is part of, or is integrated with, a smartphone that comprises, or is based on, an Apple iPhone 6 or a Samsung Galaxy S6.

### Claim 49 (depends on 45)

The method according to claim 45, further comprising storing, operating, or using an operating system, by the client device.

### Claim 50 (depends on 49)

The method according to claim 49, wherein the operating system is a mobile operating system.

### Claim 51 (depends on 50)

The method according to claim 50, wherein the mobile operating system is based on, or comprises, Android version 2.2 (Froyo), Android version 2.3 (Gingerbread), Android version 4.0 (Ice Cream Sandwich), Android Version 4.2 (Jelly Bean), Android version 4.4 (KitKat), Apple iOS version 3, Apple iOS version 4, Apple iOS version 5, Apple IOS version 6, Apple IOS version 7, Microsoft Windows® Phone version 7, Microsoft Windows® Phone version 8, Microsoft Windows® Phone version 9, or Blackberry® operating system.

### Claim 52 (depends on 1)

The method according to claim 1, further comprising storing, operating, or using, by the requesting device, the first device, the second device, or the third device, a web browser.

### Claim 53 (depends on 52)

The method according to claim 52, wherein the web browser consists of, comprises of, or is based on, Microsoft Internet Explorer, Google Chrome, Opera™, or Mozilla Firefox®.

### Claim 54 (depends on 52)

The method according to claim 52, wherein the web browser is a mobile web browser.

### Claim 55 (depends on 54)

The method according to claim 54, wherein the mobile web browser consists of, comprises of, or is based on, Safari, Opera Mini™, or Android web browser.

### Claim 56 (depends on 1)

The method according to claim 1, wherein the requesting device, the first device, the second device, or the third device, is integrated, in part or entirely, in an appliance.

### Claim 57 (depends on 56)

The method according to claim 56, wherein a primary functionality of the appliance is associated with food storage, handling, or preparation.

### Claim 58 (depends on 57)

The method according to claim 57, wherein a primary function of the appliance is heating food, and wherein the appliance is a microwave oven, an electric mixer, a stove, an oven, or an induction cooker.

### Claim 59 (depends on 57)

The method according to claim 57, wherein the appliance is a refrigerator, a freezer, a food processor, a dishwasher, a food blender, a beverage maker, a coffeemaker, or an iced-tea maker.

### Claim 60 (depends on 56)

The method according to claim 56, wherein a primary function of the appliance is associated with environmental control, and the appliance consists of, or is part of, an HVAC system.

### Claim 61 (depends on 60)

The method according to claim 60, wherein a primary function of the appliance is associated with temperature control, and wherein the appliance is an air conditioner or a heater.

### Claim 62 (depends on 56)

The method according to claim 56, wherein a primary function of the appliance is associated with cleaning, wherein the primary function is associated with clothes cleaning, and the appliance is a washing machine or a clothes dryer, or wherein the appliance is a vacuum cleaner.

### Claim 63 (depends on 56)

The method according to claim 56, wherein a primary function of the appliance is associated with water control or water heating.

### Claim 64 (depends on 56)

The method according to claim 56, wherein the appliance is an answering machine, a telephone set, a home cinema method, a HiFi method, a CD or DVD player, an electric furnace, a trash compactor, a smoke detector, a light fixture, or a dehumidifier.

### Claim 65 (depends on 56)

The method according to claim 56, wherein the appliance is a battery-operated portable electronic device, and the appliance is a notebook, a laptop computer, a media player, a cellular phone, a Personal Digital Assistant (PDA), an image processing device, a digital camera, a video recorder, a handheld computing device, or any combination thereof.

### Claim 66 (depends on 56)

The method according to claim 56, wherein the integration involves sharing a component.

### Claim 67 (depends on 66)

The method according to claim 66, wherein the integration involves housing in same enclosure, sharing same processor, or mounting onto same surface.

### Claim 68 (depends on 66)

The method according to claim 66, wherein the integration involves sharing a same connector.

### Claim 69 (depends on 68)

The method according to claim 68, wherein the connector is a power connector for connecting to a power source, and wherein the integration involves sharing the same connector for being powered from same power source, or wherein the integration involves sharing same power supply.

### Claim 70 (depends on 1)

The method according to claim 1, wherein the first device, the second device, or the third device, consists of, comprises, is integrated with, or is part of, a wearable device that is wearable on a person.

### Claim 71 (depends on 70)

The method according to claim 70, wherein the wearable device is wearable on an organ of a person head.

### Claim 72 (depends on 71)

The method according to claim 71, wherein the organ is an eye, ear, face, cheek, nose, mouth, lip, forehead, or chin.

### Claim 73 (depends on 70)

The method according to claim 70, wherein the wearable device is constructed to have a form substantially similar to, is constructed to have a shape allowing mounting or wearing identical or similar to, or is constructed to have a form to at least in part substitute for, headwear, eyewear, or earpiece.

### Claim 74 (depends on 73)

The method according to claim 73, wherein the headwear consists of, structured as, or comprises, a bonnet, a cap, a crown, a fillet, a hair cover, a hat, a helmet, a hood, a mask, a turban, a veil, or a wig.

### Claim 75 (depends on 73)

The method according to claim 73, wherein the eyewear consists of, structured as, or comprises, glasses, sunglasses, a contact lens, a blindfold, or a goggle.

### Claim 76 (depends on 73)

The method according to claim 73, wherein the earpiece consists of, structured as, or comprises, a hearing aid, a headphone, a headset, or an earplug.

### Claim 77 (depends on 70)

The method according to claim 70, wherein the wearable device is shaped for permanently or releasably being attachable to, or be part of, a clothing piece of a person.

### Claim 78 (depends on 77)

The method according to claim 77, wherein the attaching uses taping, gluing, pinning, enclosing, encapsulating, a pin, or a latch and hook clip.

### Claim 79 (depends on 78)

The method according to claim 78, wherein the clothing piece is a top, bottom, or full-body underwear, or a headwear, a footwear, an accessory, an outwear, a suit, a dress, a skirt, or a top.

### Claim 80 (depends on 70)

The method according to claim 70, wherein the wearable device further comprises an annular member defining an aperture therethrough that is sized for receipt therein of a part of a human body.

### Claim 81 (depends on 1)

The method according to claim 1, further comprising storing, operating, or using, by the requesting device, the first device, the second device, or the third device, a client operating system.

### Claim 82 (depends on 81)

The method according to claim 81, wherein the client operating system consists of, comprises, or is based on, one out of Microsoft Windows 7, Microsoft Windows XP, Microsoft Windows 8, Microsoft Windows 8.1, Linux, and Google Chrome OS.

### Claim 83 (depends on 81)

The method according to claim 81, wherein the client operating system is a Real-Time Operating System (RTOS).

### Claim 84 (depends on 83)

The method according to claim 83, wherein the RTOS comprises FreeRTOS, SafeRTOS, QNX, VxWorks, or Micro-Controller Operating Systems (uC/OS).
