System for providing database functions for multiple internet sources

A method for automatically extracting data from a target page of at least one electronic document being accessible over a computer network, said target page comprising a plurality of elements each having a contents or structural definition wherein said structural definitions interrelate said plurality of elements, said method comprising:

• recording a sequence of actions associated with an initial visit to said target page by a user operable to electronically navigate to said target page of said electronic document;
• receiving user inputs defining a user selected subset of said plurality of elements to be extracted based on at least one of said contents and structural definitions;
• generating a target pattern to identify said user selected subset of said plurality of elements to be extracted in subsequent visits to said target page;
• automatically accessing said target page according to said recorded sequence for at least one subsequent visit to said target page;
• automatically identifying and scraping a subset of select ones of said plurality of elements dependent upon said target pattern for each said at least one subsequent visit;
• remapping said target page by re-identifying any modified structural definitions of said plurality of elements thereby to enable access to an altered target page;
• and
• identifying and scraping said select ones of said plurality of elements dependent upon said target pattern and said re-identified structural definitions to thereby automatically identify and scrape said select ones of said plurality of elements from said altered target page dependent upon said target pattern;
• wherein information of interest to said user is automatically extracted from said target page for each said at least one subsequent visit to said target page.

A computerized system for automatically scraping select data from a web site, data associated with said web site including a plurality of elements each having contents or structural data associated therewith, and being stored an a server being accessible through the Internet or other computer network, said contents and structural data and elements defining a select web page or other web-accessible document of said web site, said system comprising:

• a navigation module being operable on a microprocessor-based device electronically coupled to the Internet or other computer network, said navigational module being operable to:
• record a sequence of actions of a user operable to electronically navigate to said select web page or other web-accessible document of said web site using the Internet or other computer network;
• and
• automatically access said select web page or other web-accessible document according to said recorded sequence for at least one subsequent visit to said select web page or other web-accessible document of said web site;
• and
• an extraction module being operable on said microprocessor-based device, said extraction module being operable to:
• receive user inputs identifying information of interest and generating a pattern for a select subset of said plurality of elements on said select web page or other web-accessible document for extracting said information of interest to said user;
• automatically identify and scrape select ones of said plurality of elements of said select web page, or other web-accessible document dependent upon said pattern for each said at least one subsequent visit to said select web page or other web-accessible document;
• remap said web page or other web-accessible document by re-identifying any modified structural definitions of said plurality of elements thereby to enable access to an altered web page or other web-accessible document;
• and
• identify and scrape said select ones of said plurality of elements dependent upon said pattern and said re-identified structural definitions to thereby automatically identify and scrape said select ones of said plurality of elements from said altered web page or other web-accessible document dependent upon said pattern;
• wherein information of interest to said user is automatically extracted in each said at least one subsequent visit to said target page.

A system for automatically extracting data from at least one electronic document accessible over a computer network, comprising:

• a navigation module to record a navigation path to a target page selected by a user, said target page comprising a plurality of elements each having a contents or structural definition, wherein said structural definition interrelates said plurality of elements;
• an extraction recording module receiving at least one user input for identifying information of interest on said target page to be extracted and generating a target pattern for extracting said information of interest;
• a navigation playback module to automatically access said target page according to said navigation path for at least one subsequent visit to said target page;
• an extraction playback module using said target pattern to extract said information of interest from said target page for each said at least one subsequent visit to said target page;
• said extraction recording module remapping said target page by re-identifying any modified structural definitions of said plurality of elements thereby to enable access to an altered target page;
• and
• said extraction playback module identifies and scrapes said select ones of said plurality of elements dependent upon said target pattern and said re-identified structural definitions to thereby automatically identify and scrape said select ones of said plurality of elements from said altered target page dependent upon said target pattern;
• wherein information of interest to said user is automatically extracted from said target page in each said at least one subsequent visit to said target page.

A method for automatically extracting data from an electronic document being accessible over a computer network, comprising:

• in an initial visit by a user to a target page of said electronic document, recording a navigation path to said target page and receiving at least one user input defining information of interest to said user in said target page to be extracted, said target page comprising a plurality of elements each having a contents or structural definition, wherein said structural definition interrelates said plurality of elements;
• generating a target pattern for extracting said information of interest from said target page;
• automatically accessing said target page according to said navigation path to return to said target page for at least one subsequent visit;
• for each subsequent visit to said target page, extracting information from said target page based on said target pattern, said extracting information including remapping said target page by re-identifying any modified structural definitions of plurality of elements thereby to enable access to an altered target page and identifying and scraping said select ones of said plurality of elements dependent upon said target pattern and said re-identified structural definitions to thereby automatically identify and scrape said select ones of said plurality of elements from said altered target page dependent upon said target pattern;
• wherein information of interest to said user is automatically extracted from said target page for each said at least one subsequent visit.

A computer implemented method for automatically extracting data from an electronic document being accessible over a computer network, comprising:

• providing a graphical user interface for a user to identify a web page as a target page, said target page comprising a plurality of elements each having a contents or structural definition, wherein said structural definition interrelates said plurality of elements;
• recording a navigation path to said target page;
• providing a user interface for said user to identify a subset of a plurality of elements of said target page as being information of interest to said user to be extracted;
• generating a target pattern for extracting said information of interest from said target page;
• automatically accessing said target page according to said navigation path for at least one return visit to said target page;
• for each return visit to said target page, extracting information from said target page based on said target pattern, said extracting information including remapping said target page by re-identifying any modified structural definitions of said plurality of elements thereby to enable access to an altered target page and identifying and scraping said select ones of said plurality of elements dependent upon said target pattern and said re-identified structural definitions to thereby automatically identify and scrape said select ones of said plurality of elements from said altered target page dependent upon said target pattern;
• and
• transforming said extracted information into a standard format for further processing;
• wherein information of interest to said user is automatically extracted from said target page and transformed into a format suitable for use by at least one other software application for each said at least one return visit to said target page.