Generating lossless static object models of dynamic webpages

A method for generating a static object model of a dynamic webpage,

• the method comprising:
• receiving, at a computer system, a webpage from a web server;
• instantiating, by the computer system, a headless web browser application to render the webpage, wherein the headless web browser includes browser memory and is configured to render the webpage by creating a document object model (“DOM”) and a cascading style sheet object model (“CSSOM”) in the browser memory, the DOM and the CSSOM representing dynamic rendered content of the webpage;
• injecting, by the computer system, a probe script into the headless browser, wherein the probe script is configured to retrieve the dynamic rendered content of the webpage;
• traversing, by the probe script, the DOM, including traversing regular nodes of the DOM and shadow nodes of a shadow DOM that is attached to regular nodes of the DOM;
• retrieving, by the probe script, dynamic information for the regular nodes and the shadow nodes from, at least, the DOM and CSSOM, wherein the information includes style information for the DOM and shadow style information for the shadow DOM;
• building, by the computer system, a static object model of the webpage based on the dynamic information, wherein the static object model including (i) the regular nodes, (ii) the shadow nodes, (iii) the style information for the DOM, (iv) the shadow style information, and (v) structural features delineating separate styles applying to the regular nodes and the shadow nodes;
• and
• outputting, by the computer system, the static object model for the webpage.