System and method for near real time web scraping

A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process, the process comprising:

• receiving a uniform resource locator (URL) request directed at a web server;
• detecting in the URL request a textual resource and a non-textual resource;
• generating a request for the textual resource over a first network path including a first determined latency;
• generating a request for the non-textual resource over a second network path including a second determined latency, which is greater than the first determined latency, wherein the second network path is different from the first network path;
• receiving the textual resource;
• scraping data from the textual resource prior to receiving the non-textual resource;
• and
• generating a modified web page based on the scraped data, the textual resource, and the non-textual resource.

A system for near real time scraping of a web page to generate a modified web page, comprising:

• a processing circuitry;
• and
• a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
• receive a uniform resource locator (URL) request directed at a web server;
• detect in the URL request a textual resource and a non-textual resource;
• generate a request for the textual resource over a first network path including a first determined latency;
• generate a request for the non-textual resource over a second network path including a second determined latency, which is greater than the first determined latency, wherein the second network path is different from the first network path;
• receive the textual resource;
• scrape data from the textual resource prior to receiving the non-textual resource;
• and
• generate a modified web page based on the scraped data, the textual resource, and the non-textual resource.