# US 11,909,547 B2 · System and method for improving content fetching by selecting tunnel devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,909,547 B2 - Document ID: US11909547B2 - Status: Granted - Kind: B2 - Application number: 17/672,781 - Filed: 2022-02-16 - Published: 2022-06-02 - Granted: 2024-02-20 - Earliest claimed: 2017-08-28 - Assignees: Bright Data Ltd. - Applicants: Bright Data Ltd. - Inventors: Derry Shribman; Ofer Vilenski - Web page: https://patentgrep.com/patent/US11909547B2 ## CPC - H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP] - G06F 7/58: Random or pseudo-random number generators - G06F 7/588: {Random number generators, i.e. based on natural stochastic processes} - G06F 8/71: Version control (security arrangements therefor G06F21/57); Configuration management - G06F 9/45545: {Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox} - G06F 9/4555: {Para-virtualisation, i.e. guest operating system has to be modified} - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 16/955: using information identifiers, e.g. uniform resource locators [URL] - H04L 12/2803: {Home automation networks} - H04L 12/2856: {Access arrangements, e.g. Internet access (asynchronous transfer mode networks H04L12/5601; broadband local area networks H04L12/2801; optical access or distribution networks H04Q11/0067; access to open networks H04L12/5691; digital subscriber line end-user equipment and bit-level processing of data on a PSTN-based network H04M11/00; home network gateways H04L12/2834; wireless access networks H04W)} - H04L 12/4633: {Interconnection of networks using encapsulation techniques, e.g. tunneling} - H04L 47/283: in response to processing delays, e.g. caused by jitter or round trip time [RTT] - H04L 61/256: NAT traversal - H04L 61/2575: using address mapping retrieval, e.g. simple traversal of user datagram protocol through session traversal utilities for NAT [STUN] - H04L 61/2585: through application level gateway [ALG] - H04L 61/2589: over a relay server, e.g. traversal using relay for network address translation [TURN] - H04L 61/2592: using tunnelling or encapsulation - H04L 61/4511: using domain name system [DNS] - H04L 61/5007: Internet protocol [IP] addresses - H04L 63/0272: {Virtual private networks} - H04L 63/0281: {Proxies} - H04L 63/029: {Firewall traversal, e.g. tunnelling or, creating pinholes} - H04L 63/164: {at the network layer} - H04L 67/01: Protocols - H04L 67/025: for remote control or remote monitoring of applications - H04L 67/1008: based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) - H04L 67/1021: based on client or server locations - H04L 67/133: Protocols for remote procedure calls [RPC] - H04L 67/141: Setup of application sessions (admission control or resource allocation in data switching networks H04L47/70) - H04L 67/142: Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms - H04L 67/288: Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level - H04L 67/2885: Hierarchically arranged intermediate devices, e.g. for hierarchical caching - H04L 67/56: Provisioning of proxy services (store-and-forward switching systems in data switching networks H04L12/54) - H04L 67/563: Data redirection of data network streams - H04L 67/567: Integrating service provisioning from a plurality of service providers - H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching - H04L 67/5681: Pre-fetching or pre-delivering data based on network characteristics - H04L 67/63: Routing a service request depending on the request content or context - H04L 69/16: Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] - H04L 69/162: {involving adaptations of sockets based mechanisms (secure socket layer H04L63/168)} - H04L 69/167: Adaptation for transition between two IP versions, e.g. between IPv4 and IPv6 (translation of Internet protocol [IP] addresses H04L61/2503) - H04L 69/168: specially adapted for link layer protocols, e.g. asynchronous transfer mode [ATM], synchronous optical network [SONET] or point-to-point protocol [PPP] - H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication - H04W 48/18: Selecting a network or a communication service - H04W 84/10: Small scale networks; Flat hierarchical networks - H04W 84/18: Self-organising networks, e.g. ad-hoc networks or sensor networks - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 2009/45591: {Monitoring or debugging support} - G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances} - H04L 61/2514: between local and global IP addresses - H04L 61/2553: Binding renewal aspects, e.g. using keep-alive messages - H04L 69/325: in the network layer [OSI layer 3], e.g. X.25 (H04L69/16 takes precedence) - H04L 69/326: in the transport layer [OSI layer 4] (H04L69/16 takes precedence) - H04L 69/329: in the application layer [OSI layer 7] - H04L 2101/69: using geographic information, e.g. room number - Y10S 30/01: Rechargeable battery operated - Y10S 320/21: State of charge of battery - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A method for fetching a content from a web server to a client device is disclosed, using tunnel devices serving as intermediate devices. The tunnel device is selected based on an attribute, such as IP Geolocation. A tunnel bank server stores a list of available tunnels that may be used, associated with values of various attribute types. The tunnel devices initiate communication with the tunnel bank server, and stays connected to it, for allowing a communication session initiated by the tunnel bank server. Upon receiving a request from a client to a content and for specific attribute types and values, a tunnel is selected by the tunnel bank server, and is used as a tunnel for retrieving the required content from the web server, using standard protocol such as SOCKS, WebSocket or HTTP Proxy. The client only communicates with a super proxy server that manages the content fetching scheme. ## Claims ### Claim 1 A method for obtaining a content that is identified by a content identifier and that is stored in a web server, implemented by a mobile device that is powered by a rechargeable battery, that is operative to be in idle and non-idle states and that communicates over the Internet with distinct first and second servers from a plurality of servers, the method comprising: connecting, by the mobile device, to the Internet; initiating, by the mobile device, a secure communication with the first server over an established connection over the Internet; sensing, by the mobile device, a charging level of the rechargeable battery by measuring or estimating battery voltage, current, and charge level; comparing the sensed charging level to a threshold level; sending, by the mobile device to the first server over the Internet, the sensed charging level being over or below the threshold level; responsive to being in the non-idle state, determining, when an idling condition is met; responsive to the determination that the idling condition is met, shifting to the idle state; responsive to being in the idle state, determining when the idling condition is met; responsive to the determination that the idling condition is not met, shifting to the non-idle state; periodically sending, by the mobile device to the first server or the second server, a status message responsive to the mobile device either idle or non-idle state; receiving, by the mobile device from the first or second server over the Internet, the content identifier; sending, by the mobile device to the web server over the Internet, the content identifier, in response to the receiving of the content identifier; receiving, by the mobile device from the web server over the Internet, the content, in response to the sent content identifier; and sending, by the mobile device to the first or second server over the Internet, the received content. ### Claim 2 (depends on 1) The method according to claim 1, wherein the mobile device is addressable in the Internet by a first IP address, and wherein the initiating comprises sending, by the mobile device to the first server, the first IP address. ### Claim 3 (depends on 1) The method according to claim 1, wherein the established connection is a TCP connection using ‘Active OPEN’, ‘Passive OPEN’, or TCP keepalive mechanism. ### Claim 4 (depends on 1) The method according to claim 1, wherein the established connection is based on, Virtual Private Network (VPN). ### Claim 5 (depends on 1) The method according to claim 1, wherein the communication over the Internet by the mobile device with the first or second server, is based on, uses, or is compatible with, HTTP Proxy protocol or connection, wherein the first or second server serves as an HTTP Proxy server and the mobile device serves as an HTTP Proxy client. ### Claim 6 (depends on 1) The method according to claim 1, wherein the sensing uses a Battery Management System (BMS). ### Claim 7 (depends on 1) The method according to claim 1, further comprising executing or using an operating system, and wherein the sensing comprises using the operating system. ### Claim 8 (depends on 1) The method according to claim 1, wherein the threshold level is above one of either 40%, 50%, 60%, 70%, 80%, or 90% of a battery defined full charge capacity level. ### Claim 9 (depends on 1) The method according to claim 1, wherein the mobile device is part of, or comprises, a vehicular device, or is amounted in a vehicle. ### Claim 10 (depends on 1) The method according to claim 1, wherein the mobile device is housed in a single enclosure that is a hand-held enclosure or a portable enclosure. ### Claim 11 (depends on 1) The method according to claim 1, wherein the mobile device comprises, is part of, or is integrated with, at least one of a notebook-computer, a laptop computer, a media player, a Digital Still Camera (DSC), a Digital video Camera (DVC or digital camcorder), a Personal Digital Assistant (PDA), a cellular telephone, a digital camera, or a video recorder. ### Claim 12 (depends on 1) The method according to claim 1, further comprising sending, by the mobile device to the first server, a message responsive to shifting to the idle state. ### Claim 13 (depends on 1) The method according to claim 1, further comprising sending, by the mobile device to the first server, a message responsive to shifting to the non-idle state. ### Claim 14 (depends on 1) The method according to claim 1, wherein the sensing of the charging level comprises periodically or continuously sensing of the charging level. ### Claim 15 (depends on 1) The method according to claim 1, wherein the content includes, consists of, or comprises, a part or whole of files, text, numbers, audio, voice, multimedia, video, images, music, or computer program. ### Claim 16 (depends on 1) The method according to claim 1, wherein the content includes, consists of, or comprises, a part of, or a whole of, a web-site page. ### Claim 17 (depends on 1) The method according to claim 1, wherein the message comprises, or is based on, an ‘heartbeat’ message, and wherein the time period between multiple messages sent is at least 10 milliseconds, 20 milliseconds, 30 milliseconds, 50 milliseconds, 100 milliseconds, 1 second, 2 seconds, 3 seconds, 5 seconds, 10 seconds, 20 seconds, 30 seconds, 50 seconds, or 100 seconds, 1 minute, 2 minutes, 3 minutes, minutes 5, or 10 minutes. ### Claim 18 (depends on 1) The method according to claim 1, wherein the content comprises a web-page or a web-site, and wherein the content identifier is an Uniform Resource Identifier (URI) or an Uniform Resource Locator (URL). ### Claim 19 (depends on 1) The method according to claim 1, wherein the attribute type is a geographical location, and wherein one of more attribute values comprise a name or an identifier of a continent, a country, a region, a city, a street, a ZIP code, or a timezone. ### Claim 20 (depends on 1) The method according to claim 1, for use with a Domain Name System (DNS) server, wherein the content identifier comprises a domain name, the method further comprising performing, by the mobile device using the DNS server, a DNS resolution for obtaining a numerical IP address. ### Claim 21 (depends on 1) The method according to claim 1, wherein the battery comprises a smart battery that includes plus and minus terminals, and a communication interface. ### Claim 22 (depends on 1) The method according to claim 1, wherein the sensing comprises measuring or estimating battery State of Health (SoH), total voltage, voltages of one or more individual cells, minimum and maximum cell voltage, voltage of periodic taps, temperature, average temperature, coolant intake temperature, coolant output temperature, or temperatures of one or more individual cells. ### Claim 23 (depends on 1) The method according to claim 1, wherein the receiving of the content identifier comprises receiving of the content identifier from the first server. ### Claim 24 (depends on 23) The method according to claim 23, wherein the sending of the content comprises sending the content to the first server by the web server. ### Claim 25 (depends on 23) The method according to claim 23, wherein the sending of the content comprises sending the content to the second server. ### Claim 26 (depends on 1) The method according to claim 1, wherein the receiving of the content identifier comprises receiving of the content identifier from the second server. ### Claim 27 (depends on 26) The method according to claim 26, wherein the sending of the content comprises sending the content to the second server. ### Claim 28 (depends on 1) The method according to claim 1, wherein the initiating is based on, a Network Address Translator (NAT) traversal scheme. ### Claim 29 (depends on 28) The method according to claim 28, wherein the NAT traversal scheme is according to, Internet Engineering Task Force (IETF) Request for Comments (RFC) 2663, IETF RFC 3715, IETF RFC 3947, IETF RFC 5128, IETF RFC 5245, IETF RFC 5389, or IETF RFC 7350. ### Claim 30 (depends on 28) The method according to claim 28, wherein the NAT traversal scheme is according to, Traversal Using Relays around NAT (TURN), Socket Secure (SOCKS), NAT ‘hole punching’, Session Traversal Utilities for NAT (STUN), Interactive Connectivity Establishment, (ICE), UPnP Internet Gateway Device Protocol (IGDP), or Application-Level Gateway (ALG). ### Claim 31 (depends on 1) The method according to claim 1, wherein the communication over the Internet by the mobile device with the first or second server, is based on, Transmission Control Protocol over Internet Protocol (TCP/IP) protocol or connection. ### Claim 32 (depends on 31) The method according to claim 31, wherein the communication over the Internet by the mobile device with the first or second server, is based on, uses, or is compatible with, HTTP or HTTPS protocol or connection, wherein the first or second server serves as an HTTP or HTTPS server and the mobile device serves as an HTTP or HTTPS client. ### Claim 33 (depends on 31) The method according to claim 31, wherein the communication over the Internet by the mobile device with the first or second server, is based on, or is compatible with, Socket Secure (SOCKS) protocol or connection, wherein the first or second server serves as a SOCKS server and the mobile device serves as a SOCKS client. ### Claim 34 (depends on 33) The method according to claim 33, wherein the SOCKS protocol or connection is according to, based on, or is compatible with, SOCKS4, SOCKS4a, or SOCKS5. ### Claim 35 (depends on 33) The method according to claim 33, wherein the SOCKS protocol or connection is according to, IETF RFC 1928, IETF RFC 1929, IETF RFC 1961, or IETF RFC 3089. ### Claim 36 (depends on 1) The method according to claim 1, wherein the sending of the sensed charging level is in response to the comparing. ### Claim 37 (depends on 36) The method according to claim 36, wherein the sending of the sensed charging level is in response to the sensed charging level being below or above the threshold level. ### Claim 38 (depends on 1) The method according to claim 1, wherein the mobile device comprises, is part of, or is integrated with, a smartphone. ### Claim 39 (depends on 38) The method according to claim 38, wherein the web server uses HyperText Transfer Protocol (HTTP) and responds to HTTP requests via the Internet, and wherein the sending of the content identifier to the web server comprises sending of a HTTP request that comprises the content identifier. ### Claim 40 (depends on 39) The method according to claim 39, wherein the communication with the web server is based on, or uses, HTTP persistent connection. ### Claim 41 (depends on 39) The method according to claim 39, wherein the web server uses HyperText Transfer Protocol Secure (HTTPS) and responds to HTTPS requests via the Internet, and wherein the sending of the content identifier to the web server comprises sending of a HTTPS request that comprises the content identifier. ### Claim 42 (depends on 1) The method according to claim 1, further comprising storing, operating, or using an operating system. ### Claim 43 (depends on 42) The method according to claim 42, wherein the operating system is a mobile operating system. ### Claim 44 (depends on 1) The method according to claim 1, wherein the connecting to the Internet is implemented via a wireless network. ### Claim 45 (depends on 44) The method according to claim 44, wherein the wireless network comprises, a Wireless Personal Area Network (WPAN). ### Claim 46 (depends on 1) The method according to claim 1, wherein at least part of steps are included in a Software Development Kit (SDK) that is provided as a non-transitory computer readable medium containing computer instructions, and wherein the method further comprising installing the SDK. ### Claim 47 (depends on 44) The method according to claim 44, wherein the wireless network comprises, or consists of, a Wireless Wide Area Network (WWAN). ### Claim 48 (depends on 47) The method according to claim 47, wherein the WWAN is a wireless broadband network. ### Claim 49 (depends on 48) The method according to claim 48, wherein the wireless network comprises, or consists of, a WiMAX network, and the WiMAX network is according to, compatible with, or based on, IEEE 802.16-2009. ### Claim 50 (depends on 44) The method according to claim 44, wherein the wireless network involves a cellular telephone network. ### Claim 51 (depends on 50) The method according to claim 50, wherein the cellular telephone network is a Third Generation (3G) network that uses a protocol selected from the group consisting of UMTS W-CDMA, UMTS HSPA, UMTS TDD, CDMA2000 1×RTT, CDMA2000 EV-DO, and GSM EDGE-Evolution, or wherein the cellular telephone network uses a protocol selected from the group consisting of a Fourth Generation (4G) network that uses HSPA+, Mobile WiMAX, LTE, LTE-Advanced, MBWA, or is based on IEEE 802.20-2008. ### Claim 52 (depends on 44) The method according to claim 44, wherein the wireless network comprises, a Wireless Local Area Network (WLAN). ### Claim 53 (depends on 52) The method according to claim 52, wherein the WLAN is compatible with, a standard selected from the group consisting of IEEE 802.11-2012, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and IEEE 802.11ac. ### Claim 54 (depends on 1) The method according to claim 1, further comprising storing, operating, or using, by the mobile device, a web browser. ### Claim 55 (depends on 54) The method according to claim 54, wherein the web browser is a mobile web browser. ### Claim 56 (depends on 1) The method according to claim 1, wherein the mobile device is associated with a single IP address. ### Claim 57 (depends on 1) The method according to claim 1, wherein the mobile device is associated with multiple IP addresses. ### Claim 58 (depends on 1) The method according to claim 1, further comprising operating, by the mobile device, an operating system or a program process or thread, and wherein the idling condition is determined to be met based on, or according to, activating or executing the process or thread by the operating system or the program. ### Claim 59 (depends on 58) The method according to claim 58, wherein the process or thread comprises a low-priority or background task, an idle process, or a screensaver. ### Claim 60 (depends on 58) The method according to claim 58, wherein the process or thread comprises using an entire screen for displaying. ### Claim 61 (depends on 58) The method according to claim 58, wherein the mobile device comprises a network interface or a network transceiver for communication over a network, the method further comprising metering, an amount of data transmitted to, or received from, the network during a time interval, and wherein the idling condition is determined to be met based on, or according to, the metered amount of data being under the threshold level. ### Claim 62 (depends on 58) The method according to claim 58, wherein the idling condition is determined to be met based on, or according to, the sensed charge level being over or below the threshold level. ### Claim 63 (depends on 58) The method according to claim 58, further comprising monitoring or metering, a resource utilization, and wherein the idling condition is determined to be met based on, or according to, the monitored or metered resource utilization being under the threshold. ### Claim 64 (depends on 63) The method according to claim 63, wherein the resource utilization comprises the utilization or a processor in the mobile device. ### Claim 65 (depends on 58) The method according to claim 58, wherein the mobile device comprises an input device for obtaining an input from a human user or operator, the method further comprising sensing, using the input device, the input, and wherein the idling condition is determined to be met based on, or according to, not receiving an input from the input device for a pre-set time interval. ### Claim 66 (depends on 65) The method according to claim 65, wherein the input device comprises a pointing device, a keyboard, a touchscreen, or a microphone. ### Claim 67 (depends on 58) The method according to claim 58, wherein the mobile device comprises a motion sensor for sensing motion, acceleration, vibration, or location change of the mobile device, the method further comprising sensing, using the motion sensor, the mobile device motion, acceleration, vibration, or location change, and wherein the idling condition is determined to be met based on, or according to, respectively sensing the motion, the vibration, the acceleration, or the location change being under the threshold. ### Claim 68 (depends on 67) The method according to claim 67, wherein the motion sensor comprises an accelerometer, gyroscope, vibration sensor, or a Global Positioning System (GPS) receiver. ### Claim 69 (depends on 1) The method according to claim 1, further comprising sending, by the mobile device to the first server, a first value of a first attribute type. ### Claim 70 (depends on 69) The method according to claim 69, wherein the first attribute type comprises Internet Service Provider (ISP) or Autonomous System Number (ASN). ### Claim 71 (depends on 70) The method according to claim 70, wherein the first value comprises a name or an identifier of the ISP or the ASN number. ### Claim 72 (depends on 69) The method according to claim 69, wherein the first attribute type corresponds to a hardware of software of the mobile device. ### Claim 73 (depends on 72) The method according to claim 72, wherein the first attribute type comprises the hardware of the mobile device. ### Claim 74 (depends on 73) The method according to claim 73, wherein the first values comprise stationary and portable values, respectively based on the mobile device being stationary or portable. ### Claim 75 (depends on 72) The method according to claim 72, wherein the first attribute type comprises a software application installed, used, or operated, in the mobile device. ### Claim 76 (depends on 75) The method according to claim 75, wherein the first values comprise the type, make, model, or version of the software. ### Claim 77 (depends on 75) The method according to claim 75, wherein the software comprises an operating system. ### Claim 78 (depends on 69) The method according to claim 69, wherein the first attribute type corresponds to a communication property, feature of a communication link to the Internet of the mobile device. ### Claim 79 (depends on 78) The method according to claim 78, wherein the communication link corresponds to the communication link with the web server, the first server, or the second server. ### Claim 80 (depends on 78) The method according to claim 78, wherein the first attribute type corresponds to a bandwidth (BW) or Round-Trip delay Time (RTT) of the communication link, and the first value is the respective estimation or measurement of the BW or RTT. ### Claim 81 (depends on 80) The method according to claim 80, further comprising estimating or measuring, by the mobile device, the BW or RTT of the communication link. ### Claim 82 (depends on 1) The method according to claim 1, wherein the first server is randomly selected from the plurality of servers. ### Claim 83 (depends on 82) The method according to claim 82, wherein the first server is randomly selected using one or more random numbers generated by a random number generator. ### Claim 84 (depends on 83) The method according to claim 83, wherein the random number generator is hardware based. ### Claim 85 (depends on 84) The method according to claim 84, wherein the random number generator is using thermal noise, shot noise, nuclear decaying radiation, photoelectric effect, or quantum phenomena. ### Claim 86 (depends on 83) The method according to claim 83, wherein the random number generator is software based. ### Claim 87 (depends on 86) The method according to claim 86, wherein the random number generator is based on, or comprises, executing an algorithm for generating pseudo-random numbers. ### Claim 88 (depends on 1) The method according to claim 1, wherein the one of more attribute values is based on actual geographical location or on IP geolocation. ### Claim 89 (depends on 88) The method according to claim 88, wherein the geolocation is based on W3C Geolocation API.