# US 11,115,230 B2 · System and method for improving content fetching by selecting tunnel devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,115,230 B2 - Document ID: US11115230B2 - Status: Granted - Kind: B2 - Application number: 16/826,206 - Filed: 2020-03-21 - Published: 2020-10-29 - Granted: 2021-09-07 - Earliest claimed: 2017-08-28 - Assignees: Bright Data Ltd. - Applicants: Luminati Networks Ltd. - Inventors: Derry Shribman; Ofer Vilenski - Web page: https://patentgrep.com/patent/US11115230B2 ## CPC - H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP] - G06F 7/58: Random or pseudo-random number generators - G06F 7/588: {Random number generators, i.e. based on natural stochastic processes} - G06F 8/71: Version control (security arrangements therefor G06F21/57); Configuration management - G06F 9/45545: {Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox} - G06F 9/4555: {Para-virtualisation, i.e. guest operating system has to be modified} - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 16/955: using information identifiers, e.g. uniform resource locators [URL] - H04L 12/2803: {Home automation networks} - H04L 12/2856: {Access arrangements, e.g. Internet access (asynchronous transfer mode networks H04L12/5601; broadband local area networks H04L12/2801; optical access or distribution networks H04Q11/0067; access to open networks H04L12/5691; digital subscriber line end-user equipment and bit-level processing of data on a PSTN-based network H04M11/00; home network gateways H04L12/2834; wireless access networks H04W)} - H04L 12/4633: {Interconnection of networks using encapsulation techniques, e.g. tunneling} - H04L 47/283: in response to processing delays, e.g. caused by jitter or round trip time [RTT] - H04L 61/256: NAT traversal - H04L 61/2575: using address mapping retrieval, e.g. simple traversal of user datagram protocol through session traversal utilities for NAT [STUN] - H04L 61/2585: through application level gateway [ALG] - H04L 61/2589: over a relay server, e.g. traversal using relay for network address translation [TURN] - H04L 61/2592: using tunnelling or encapsulation - H04L 61/4511: using domain name system [DNS] - H04L 61/5007: Internet protocol [IP] addresses - H04L 63/0272: {Virtual private networks} - H04L 63/0281: {Proxies} - H04L 63/029: {Firewall traversal, e.g. tunnelling or, creating pinholes} - H04L 63/164: {at the network layer} - H04L 67/01: Protocols - H04L 67/025: for remote control or remote monitoring of applications - H04L 67/1008: based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) - H04L 67/1021: based on client or server locations - H04L 67/133: Protocols for remote procedure calls [RPC] - H04L 67/141: Setup of application sessions (admission control or resource allocation in data switching networks H04L47/70) - H04L 67/142: Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms - H04L 67/288: Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level - H04L 67/2885: Hierarchically arranged intermediate devices, e.g. for hierarchical caching - H04L 67/56: Provisioning of proxy services (store-and-forward switching systems in data switching networks H04L12/54) - H04L 67/563: Data redirection of data network streams - H04L 67/567: Integrating service provisioning from a plurality of service providers - H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching - H04L 67/5681: Pre-fetching or pre-delivering data based on network characteristics - H04L 67/63: Routing a service request depending on the request content or context - H04L 69/16: Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] - H04L 69/162: {involving adaptations of sockets based mechanisms (secure socket layer H04L63/168)} - H04L 69/167: Adaptation for transition between two IP versions, e.g. between IPv4 and IPv6 (translation of Internet protocol [IP] addresses H04L61/2503) - H04L 69/168: specially adapted for link layer protocols, e.g. asynchronous transfer mode [ATM], synchronous optical network [SONET] or point-to-point protocol [PPP] - H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication - H04W 48/18: Selecting a network or a communication service - H04W 84/10: Small scale networks; Flat hierarchical networks - H04W 84/18: Self-organising networks, e.g. ad-hoc networks or sensor networks - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 2009/45591: {Monitoring or debugging support} - G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances} - H04L 61/2514: between local and global IP addresses - H04L 61/2553: Binding renewal aspects, e.g. using keep-alive messages - H04L 69/325: in the network layer [OSI layer 3], e.g. X.25 (H04L69/16 takes precedence) - H04L 69/326: in the transport layer [OSI layer 4] (H04L69/16 takes precedence) - H04L 69/329: in the application layer [OSI layer 7] - H04L 2101/69: using geographic information, e.g. room number - Y10S 30/01: Rechargeable battery operated - Y10S 320/21: State of charge of battery - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A method for fetching a content from a web server to a client device is disclosed, using tunnel devices serving as intermediate devices. The tunnel device is selected based on an attribute, such as IP Geolocation. A tunnel bank server stores a list of available tunnels that may be used, associated with values of various attribute types. The tunnel devices initiate communication with the tunnel bank server, and stays connected to it, for allowing a communication session initiated by the tunnel bank server. Upon receiving a request from a client to a content and for specific attribute types and values, a tunnel is selected by the tunnel bank server, and is used as a tunnel for retrieving the required content from the web server, using standard protocol such as SOCKS, WebSocket or HTTP Proxy. The client only communicates with a super proxy server that manages the content fetching scheme. ## Claims ### Claim 1 A method for fetching a content identified by a content identifier from a web server by using an appliance that serves as an HTTP Proxy client and that is operating in multiple states that includes an idle state and non-idle states, for use with a plurality of HTTP Proxy servers, that includes first and second servers, each of the plurality of servers is connectable to the Internet and is addressable in the Internet using a respective IP address, the first server stores a list of IP addresses, and the appliance that are each connected to the Internet and are each addressable in the Internet using a respective IP address, the method by the appliance comprising: storing, operating, or using an operating system, a program process, or a thread; monitoring or metering, a resource utilization; responsive to being in one of the non-idle states, determining, if an idling condition is met; responsive to the determination that the idling condition is met, shifting to the idle state; responsive to being in the idle state, determining if an idling condition is met; and responsive to the determination that the idling condition is not met, shifting to one of the non-idle states; receiving, from the first or second server using HTTP Proxy protocol or connection, a first message that comprises the content identifier; in response to the receiving of the first message, initiating a communication, with the second server; randomly selecting the first server from the plurality of servers using one or more random numbers generated by a random number generator; sending, to the selected first server, a second message; sending, to the web server, a content request that comprises the content identifier; receiving, from the web server, the content, in response to the content request; and sending, to the first or second server using HTTP Proxy protocol or connection, the content, in response to the first message, wherein the communication over the Internet with the first or second server uses Socket Secure (SOCKS) protocol or connection, wherein the first or second server serves as an SOCKS server and the appliance serves as an SOCKS client, wherein the idling condition is determined to be met based on, or according to, activating or executing the process or thread by the operating system or the program, wherein the idling condition is determined to be met based on, or according to, the monitored or metered resource utilization being under a threshold, wherein the appliance is associated with a first value that comprises a first numeric value or a first identifier of a feature, a characteristic, or a property of a first attribute type, and wherein the appliance is associated with a second value that comprises a second numeric value or a second identifier of a feature, a characteristic, or a property of a second attribute type. ### Claim 2 (depends on 1) The method according to claim 1, wherein the sending, to the first or second server of the content comprises exclusively sending, to the first server, the content; or exclusively sending, to the second server, the content. ### Claim 3 (depends on 1) The method according to claim 1, wherein the first message comprises the IP address of the second server. ### Claim 4 (depends on 1) The method according to claim 1, wherein the initiating of the communication uses a Network Address Translator (NAT) traversal scheme. ### Claim 5 (depends on 1) The method according to claim 1, wherein the NAT traversal scheme uses Internet Engineering Task Force (IETF) Request for Comments (RFC) 2663, IETF RFC 3715, IETF RFC 3947, IETF RFC 5128, IETF RFC 5245, IETF RFC 5389, or IETF RFC 7350. ### Claim 6 (depends on 1) The method according to claim 1, wherein the NAT traversal scheme uses Traversal Using Relays around NAT (TURN), Socket Secure (SOCKS), NAT ‘hole punching’, Session Traversal Utilities for NAT (STUN), Interactive Connectivity Establishment, (ICE), UPnP Internet Gateway Device Protocol (IGDP), or Application-Level Gateway (ALG). ### Claim 7 (depends on 1) The method according to claim 1, wherein the communication over the Internet with the first or second server uses Transmission Control Protocol over Internet Protocol (TCP/IP) protocol or connection. ### Claim 8 (depends on 1) The method according to claim 1, wherein the communication over the Internet with the first or the second server uses HTTP or HTTPS protocol or connection, and wherein the first or second server serves as an HTTP or HTTPS server and the appliance respectively serves as an HTTP or HTTPS client. ### Claim 9 (depends on 1) The method according to claim 1, wherein the SOCKS protocol or connection is compatible with SOCKS4, SOCKS4a, or SOCKS5. ### Claim 10 (depends on 1) The method according to claim 1, wherein the SOCKS protocol or connection is compatible with IETF RFC 1928, IETF RFC 1929, IETF RFC 1961, or IETF RFC 3089. ### Claim 11 (depends on 1) The method according to claim 1, further comprising sending, to the first server, a message responsive to the appliance being in idle or non-idle state. ### Claim 12 (depends on 1) The method according to claim 1, further comprising: sending, to the first server, a first status message in response to shifting to the idle state; and sending, to the first server, a second status message in response to shifting to a non-idle state. ### Claim 13 (depends on 1) The method according to claim 1, wherein the process or thread comprises a low-priority or background task, an idle process, or a screensaver. ### Claim 14 (depends on 1) The method according to claim 1, wherein the process or thread comprises using the entire screen for displaying. ### Claim 15 (depends on 1) The method according to claim 1, wherein the resource utilization comprises the utilization or a processor in the appliance. ### Claim 16 (depends on 1) The method according to claim 1, wherein the appliance comprises a network interface or a network transceiver for communication over a network, the method further comprising metering, an amount of data transmitted to, or received from, the network during a time interval, and wherein the idling condition is determined to be met based on, or according to, the metered amount of data being under a threshold level. ### Claim 17 (depends on 1) The method according to claim 1, further comprising sending, to the first server, the first value to the first server. ### Claim 18 (depends on 1) The method according to claim 1, wherein the method further comprising, sending, to the first server, the second value. ### Claim 19 (depends on 1) The method according to claim 1, wherein the first attribute type comprises a geographical location, and wherein each of the first values comprises a name or an identifier of a continent, a country, a region, a city, a street, a ZIP code, or a timezone. ### Claim 20 (depends on 1) The method according to claim 1, wherein each of the plurality of servers is associated with a one of more attribute values relating to an attribute type, and wherein the first server is selected from the plurality of servers based on, or according to, the respective one of more attribute values. ### Claim 21 (depends on 1) The method according to claim 1, for use with a Domain Name System (DNS) server, wherein the content identifier comprises a domain name, the method further comprising performing, using the DNS server, a DNS resolution for obtaining a numerical IP address, and wherein the first message or the content request comprises the obtained numerical IP address. ### Claim 22 (depends on 1) The method according to claim 1, wherein the content comprises a web-page or a web-site, and wherein the content identifier is Uniform Resource Identifier (URI) or Uniform Resource Locator (URL). ### Claim 23 (depends on 1) The method according to claim 1, wherein each of the IP addresses is in IPv4 or IPv6 form. ### Claim 24 (depends on 1) The method according to claim 1, wherein the web server uses HyperText Transfer Protocol (HTTP) or HTTP Secure (HTTPS) for responding to respective HTTP or HTTPS requests via the Internet, and wherein the content request is an HTTP or an HTTPS request. ### Claim 25 (depends on 1) The method according to claim 1, wherein the communication over the Internet with the first or the second server, is based on, uses, or is compatible with, Transmission Control Protocol over Internet Protocol (TCP/IP) protocol or connection. ### Claim 26 (depends on 1) The method according to claim 1, wherein the appliance is associated with a single IP address from the list. ### Claim 27 (depends on 1) The method according to claim 1, wherein a primary function of the appliance is cleaning, wherein the primary function is associated with clothes cleaning, and the appliance is a washing machine or a clothes dryer, or wherein the appliance is a vacuum cleaner. ### Claim 28 (depends on 1) The method according to claim 1, wherein a primary function of the appliance is associated with water control or water heating. ### Claim 29 (depends on 1) The method according to claim 1, wherein the appliance is an answering machine, a telephone set, a home cinema method, a HiFi method, a CD or DVD player, an electric furnace, a trash compactor, a smoke detector, a light fixture, or a dehumidifier. ### Claim 30 (depends on 1) The method according to claim 1, wherein the appliance is a battery-operated portable electronic device, and the appliance is a notebook, a laptop computer, a media player, a cellular phone, a Personal Digital Assistant (PDA), an image processing device, a digital camera, a video recorder, or a handheld computing device. ### Claim 31 (depends on 1) The method according to claim 1, wherein the operating system is a mobile operating system. ### Claim 32 (depends on 1) The method according to claim 1, wherein the mobile operating system is based on, compatible with, or comprises, Android version 2.2 (Froyo), Android version 2.3 (Gingerbread), Android version 4.0 (Ice Cream Sandwich), Android Version 4.2 (Jelly Bean), Android version 4.4 (KitKat), Apple iOS version 3, Apple iOS version 4, Apple iOS version 5, Apple iOS version 6, Apple iOS version 7, Microsoft Windows® Phone version 7, Microsoft Windows® Phone version 8, Microsoft Windows® Phone version 9, or Blackberry® operating system. ### Claim 33 (depends on 1) A non-transitory computer readable medium containing computer instructions that, when executed by a computer processor, cause the processor to perform the method according to claim 1. ### Claim 34 (depends on 1) The method according to claim 1, further included in a Software Development Kit (SDK) that is provided as a non-transitory computer readable medium containing computer instructions, and wherein the method further comprising installing the SDK. ### Claim 35 (depends on 1) The method according to claim 1, wherein the appliance is addressable in the Internet using a first IP address, the method further comprising sending, to the first server, a second message that comprises at least one value with one attribute type associated with the appliance. ### Claim 36 (depends on 35) The method according to claim 35, further comprising establishing a connection with the first server, and wherein the method further comprising responding, to a communication initiated by the first server using the established connection. ### Claim 37 (depends on 36) The method according to claim 36, wherein the established connection is a TCP connection using a three-way handshake that involves ‘Active OPEN’, ‘Passive OPEN’, or TCP keepalive mechanism. ### Claim 38 (depends on 36) The method according to claim 36, wherein the established connection uses Virtual Private Network (VPN). ### Claim 39 (depends on 1) The method according to claim 1, wherein the appliance comprises an input device for obtaining an input from a human user or operator, the method further comprising sensing, using the input device, the input, and wherein the idling condition is determined to be met based on, or according to, not receiving an input from the input device for a pre-set time interval. ### Claim 40 (depends on 39) The method according to claim 39, wherein the input device comprises a pointing device, a keyboard, a touchscreen, or a microphone. ### Claim 41 (depends on 1) The method according to claim 1, wherein the appliance comprises a motion sensor for sensing motion, acceleration, vibration, or location change of the appliance, the method further comprising sensing, using the motion sensor, the appliance motion, acceleration, vibration, or location change, and wherein the idling condition is determined to be met based on, or according to, respectively sensing the motion, the vibration, the acceleration, or the location change being under a threshold. ### Claim 42 (depends on 41) The method according to claim 41, wherein the motion sensor comprises an accelerometer, gyroscope, vibration sensor, or a Global Positioning System (GPS) receiver. ### Claim 43 (depends on 1) The method according to claim 1, wherein the appliance comprises a battery, the method further comprising metering or sensing, a battery charging level, and wherein the idling condition is determined to be met based on, or according to, the metered or sensed charge level being over a threshold level. ### Claim 44 (depends on 43) The method according to claim 43, wherein the metering or sensing uses a Battery Management System (BMS). ### Claim 45 (depends on 43) The method according to claim 43, wherein the threshold level of the metered or sensed charge level is above either 40%, 50%, 60%, 70%, 80%, or 90% of the battery defined full charge capacity. ### Claim 46 (depends on 1) The method according to claim 1, wherein the first value is based on IP geolocation. ### Claim 47 (depends on 46) The method according to claim 46, wherein the geolocation is based on W3C Geolocation API. ### Claim 48 (depends on 46) The method according to claim 46, for use with a database associating IP addresses to geographical locations. ### Claim 49 (depends on 1) The method according to claim 1, wherein the first attribute type comprises Internet Service Provider (ISP) or Autonomous System Number (ASN). ### Claim 50 (depends on 49) The method according to claim 49, wherein the first value comprises a name or an identifier of the ISP or the ASN number. ### Claim 51 (depends on 1) The method according to claim 1, wherein the first attribute type corresponds to a hardware of software of the appliance. ### Claim 52 (depends on 51) The method according to claim 51, wherein the first attribute type comprises the hardware of the appliance. ### Claim 53 (depends on 52) The method according to claim 52, wherein the first values comprise stationary or portable values, respectively based on the appliance being stationary or portable. ### Claim 54 (depends on 51) The method according to claim 51, wherein the first attribute type comprises a software application that is used by the appliance. ### Claim 55 (depends on 54) The method according to claim 54, wherein the first values comprise the type, make, model, or version of the software. ### Claim 56 (depends on 54) The method according to claim 54, wherein the software comprises an operating system. ### Claim 57 (depends on 1) The method according to claim 1, wherein the first attribute type corresponds to a communication property, feature of a communication link of the appliance. ### Claim 58 (depends on 57) The method according to claim 57, wherein the first attribute type corresponds to a bandwidth (BW) or Round-Trip delay Time (RTT) of the communication link, and the first value is the respective estimation or measurement of the BW or RTT. ### Claim 59 (depends on 58) The method according to claim 58, further comprising estimating or measuring, the BW or RTT of the communication link. ### Claim 60 (depends on 57) The method according to claim 57, wherein the first attribute type corresponds to the technology or scheme used by the appliance for connecting to the Internet. ### Claim 61 (depends on 60) The method according to claim 60, wherein the first values comprise wired or wireless values, respectively based on the appliance being connected to the Internet using wired or wireless connection. ### Claim 62 (depends on 1) The method according to claim 1, wherein the random number generator is hardware based. ### Claim 63 (depends on 62) The method according to claim 62, wherein the random number generator is using thermal noise, shot noise, nuclear decaying radiation, photoelectric effect, or quantum phenomena. ### Claim 64 (depends on 1) The method according to claim 1, wherein the random number generator is software based. ### Claim 65 (depends on 64) The method according to claim 64, wherein the random number generator is based on executing an algorithm for generating pseudo-random numbers. ### Claim 66 (depends on 1) The method according to claim 1, wherein the appliance is associated with multiple IP addresses from the list. ### Claim 67 (depends on 66) The method according to claim 66, wherein the appliance is associated with more than 1,000, 2,000, 5,000, 10,000, 20,000, 50,000 or 100,000 multiple distinct IP addresses from the list. ### Claim 68 (depends on 1) The method according to claim 1, further comprising storing, operating, or using, a client operating system. ### Claim 69 (depends on 68) The method according to claim 68, wherein the client operating system consists or, comprises of, or is based on, one out of Microsoft Windows 7, Microsoft Windows XP, Microsoft Windows 8, Microsoft Windows 8.1, Linux, and Google Chrome OS. ### Claim 70 (depends on 1) The method according to claim 1, further comprising storing, operating, or using, a web browser. ### Claim 71 (depends on 70) The method according to claim 70, wherein the web browser consists of, comprises, or is compatible with, Microsoft Internet Explorer, Google Chrome, Opera™, or Mozilla Firefox®. ### Claim 72 (depends on 70) The method according to claim 70, wherein the web browser is a mobile web browser. ### Claim 73 (depends on 72) The method according to claim 72, wherein the mobile web browser consists of, comprises, or is based on, Safari, Opera Mini™, or Android web browser. ### Claim 74 (depends on 1) The method according to claim 1, wherein a primary functionality of the appliance is food storage, handling, or preparation. ### Claim 75 (depends on 74) The method according to claim 74, wherein a primary function of the appliance is heating food, and wherein the appliance is a microwave oven, an electric mixer, a stove, an oven, or an induction cooker. ### Claim 76 (depends on 74) The method according to claim 74, wherein the appliance is a refrigerator, a freezer, a food processor, a dishwasher, a food blender, a beverage maker, a coffeemaker, or an iced-tea maker. ### Claim 77 (depends on 1) The method according to claim 1, wherein a primary function of the appliance is environmental control, and the appliance consists of, or is part of, an HVAC method. ### Claim 78 (depends on 77) The method according to claim 77, wherein a primary function of the appliance is temperature control, and wherein the appliance is an air conditioner or a heater. ### Claim 79 (depends on 1) The method according to claim 1, wherein connecting to the Internet uses a wireless network that comprises a wireless broadband network. ### Claim 80 (depends on 79) The method according to claim 79, wherein the wireless network comprises, or consists of, a WiMAX network, and the WiMAX network is according to, compatible with, or based on, IEEE 802.16-2009. ### Claim 81 (depends on 79) The method according to claim 79, wherein the wireless network comprises, or consists of, a cellular telephone network. ### Claim 82 (depends on 79) The method according to claim 79, wherein the cellular telephone network is a Third Generation (3G) network that uses a protocol selected from the group consisting of UMTS W-CDMA, UMTS HSPA, UMTS TDD, CDMA2000 1×RTT, CDMA2000 EV-DO, and GSM EDGE-Evolution, or wherein the cellular telephone network uses a protocol selected from the group consisting of a Fourth Generation (4G) network that uses HSPA+, Mobile WiMAX, LTE, LTE-Advanced, MBWA, or is based on IEEE 802.20-2008. ### Claim 83 (depends on 79) The method according to claim 79, wherein the wireless network comprises, or consists of, a Wireless Personal Area Network (WPAN). ### Claim 84 (depends on 83) The method according to claim 83, wherein the WPAN is according to, compatible with, or based on, Bluetooth™, Bluetooth Low Energy (BLE), or IEEE 802.15.1-2005 standards, or wherein the WPAN is a wireless control network that is according to, or based on, Zigbee™, IEEE 802.15.4-2003, or Z-Wave™ standards. ### Claim 85 (depends on 79) The method according to claim 79, wherein the wireless network comprises, or consists of, a Wireless Local Area Network (WLAN). ### Claim 86 (depends on 85) The method according to claim 85, wherein the WLAN is compatible with a standard selected from the group consisting of IEEE 802.11-2012, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and IEEE 802.11ac.