Method for scalable, fast normalization of XML documents for insertion of data into a relational database

A method of transferring data from a markup language file having a hierarchical structure to a relational database, said hierarchical structure comprising a tree or forest of nodes on which depth first search imposes a total ordering, with some nodes designated as repeating nodes, and said method comprising:

• partitioning said hierarchical structure into sections, wherein each section is dedicated to at least one leaf node of said hierarchical structure, and wherein two non-repeating leaf nodes that are adjacent in frontier order and have the same parent are contained in the same section, frontier order being the order in which leaf nodes are encountered in a depth first search of said hierarchical structure;
• allocating a memory section for each said section of said hierarchical structure according to the data types of the nodes in the section;
• after completing said partitioning and allocating, parsing said markup language file to produce a stream of data pairs, wherein each of said data pairs comprises an element of node data and an element of node location information, and wherein said node location information indicates the location of the corresponding node within said hierarchical structure;
• loading said node data into corresponding sections as said node data elements are output from said parsing process;
• and
• transferring said node data from said sections to said relational database, wherein information is transferred from one section as soon as said loading process completes loading at least one element of node data to said one memory section and an end of section indicator has been encountered by said parsing process.

A method of transferring data from a markup language file having a hierarchical structure to a relational database, said hierarchical structure comprising a tree or forest of nodes on which depth first search imposes a total ordering, with some nodes designated as repeating nodes, and said method comprising:

• partitioning said hierarchical structure into sections, wherein each section is dedicated to at least one leaf node of said hierarchical structure, and wherein two non-repeating leaf nodes that are adjacent in frontier order and have the same parent are contained in the same section, frontier order being the order in which leaf nodes are encountered in a depth first search of said hierarchical structure;
• allocating a memory section for each said section of said hierarchical structure according to the data types of the nodes in the section;
• after completing said partitioning and allocating, parsing said markup language file to produce a stream of data pairs, wherein each of said data pairs comprises an element of node data and an element of node location information, and wherein said node location information indicates the location of the corresponding node within said hierarchical structure;, wherein each of said data pairs is in the form (tag, field), and wherein said field represents node data and said tag represents the location of corresponding node data within said hierarchical structure;
• loading said data pairs into corresponding sections as said data pairs are output from said parsing process;
• and
• transferring said node data from said sections to said relational database, wherein information is transferred from one section as soon as said loading process completes loading at least one element of node data to said one memory section and begins loading a different element of node data to a different memory section.

A method of transferring data from a markup language file having a hierarchical structure to a relational database said method comprising:

• partitioning said hierarchical structure into sections;
• allocating a memory section for each of said sections of said hierarchical structure according to the data types of the nodes in the section;
• after completing said partitioning and allocating, parsing said markup language file to produce a stream of data pairs while performing said parsing process, loading said node data into the memory section allocated for the section containing the corresponding node location as said data pairs are output from said parsing process;
• and
• transferring said node data from said sections to said relational database.

A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform a method of transferring data from a markup language file having a hierarchical structure to a relational database, said hierarchical structure comprising a tree or forest of nodes on which depth first search imposes a total ordering, with some nodes designated as repeating nodes, and said method comprising:

• partitioning said hierarchical structure into sections, wherein each section is dedicated to at least one leaf node of said hierarchical structure, and wherein two non-repeating leaf nodes that are adjacent in frontier order and have the same parent are contained in the same section, frontier order being the order in which leaf nodes are encountered in a depth first search of said hierarchical structure;
• allocating a memory section for each said section of said hierarchical structure according to the data types of the nodes in the section;
• after completing said partitioning and allocating, parsing said markup language file to produce a stream of data pairs, wherein each of said data pairs comprises an element of node data and an element of node location information, and wherein said node location information indicates the location of the corresponding node within said hierarchical structure;
• while performing said parsing process, loading said node data into the memory section allocated for the section containing the corresponding node location as said data pairs are output from said parsing process;
• and
• transferring said node data from said sections to said relational database, wherein information is transferred from one section as soon as said loading process completes loading at least one element of node data to said one memory section and begins loading a different element of node data to a different memory section.