Configuring web crawler to extract web page information

A system for configuring web crawling, comprising:

• one or more processors to:
• obtain a webpage comprising a plurality of nodes;
• present the webpage to a user;
• receive a user selection of a node in the webpage, the node comprising at least one element;
• in response to the user selection of the node, present a web crawling configuration option pertaining to a web crawling action to be performed with respect to the node, the web crawling configuration option depending at least in part on a type of an element included in the node;
• receive a user input specifying the web crawling configuration options pertaining to the web crawling action to be performed with respect to the node and a user indication to specify repeatedly the web crawling action to be performed on the plurality of nodes;
• specify repeatedly the web crawling action to be performed for the plurality of nodes;
• and
• store user specified web crawling configuration options, perform the web crawling action on the plurality of nodes according to the user input, or both;
• and
• one or more memories coupled to the one or more processors, to provide the one or more processors with instructions.

A method for configuring web crawling, comprising:

• obtaining, using one or more computer processors, a webpage comprising a plurality of nodes;
• presenting the webpage to a user;
• receiving a user selection of a node in the webpage, the node comprising at least one element;
• in response to the user selection of the node, presenting a web crawling configuration option pertaining to a web crawling action to be performed with respect to the node, the web crawling configuration option depending at least in part on a type of an element included in the node;
• receiving a user input specifying the web crawling configuration options pertaining to the web crawling action to be performed with respect to the node and a user indication to specify repeatedly the web crawling action to be performed on the plurality of nodes;
• specifying repeatedly the web crawling action to be performed for the plurality of nodes;
• and
• storing user specified web crawling configuration options, performing the web crawling action on the plurality of nodes according to the user input, or both.

A computer program product for configuring web crawling, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:

• obtaining, using one or more computer processors, a webpage comprising a plurality of nodes;
• presenting the webpage to a user;
• receiving a user selection of a node in the webpage, the node comprising at least one element;
• in response to the user selection of the node, presenting a web crawling configuration option pertaining to a web crawling action to be performed with respect to the node, the web crawling configuration option depending at least in part on a type of an element included in the node;
• receiving a user input specifying the web crawling configuration options pertaining to the web crawling action to be performed with respect to the node and a user indication to specify repeatedly the web crawling action to be performed on the plurality of nodes;
• specifying repeatedly the web crawling action to be performed for the plurality of nodes;
• and
• storing user specified web crawling configuration options, performing the web crawling action on the plurality of nodes according to the user input, or both.