Methodology supported business intelligence (BI) software and system

A method to transform raw electronic data into meaningful and useful information in a database server, comprising:

• idealizing metadata from at least one data source into a relational model, comprising, importing metadata into a repository connected to a product, generating intuitive table and column names by mapping a friendly name to an original name by the product, refining the metadata to include table keys and relationships even if this information is not accessible in the data source;
• importing data from the at least one data source to a staging data store for temporary storage;
• importing at least one table primary key from the staging data store to a surrogate data store creating a surrogate key table, wherein the surrogate data store converts all original keys and foreign key references to surrogate keys, an original key being mapped to a surrogate key, the surrogate key table reflecting the link between the original and surrogate keys, wherein during insert and update operations the surrogate key tables are used to create and maintain surrogate keys;
• processing the table for extraction to an operational data store, wherein the table updates the surrogate key table if necessary before processing, the table being updated during processing if a record with an actual surrogate primary key exists in the operational data store, the table being loaded if a record with the actual surrogate primary key does not exist in the operational data store;
• importing data to said operational data store, wherein the table has to successfully update the corresponding surrogate key table and the surrogate key table of any related tables before processing;
• and
• performing a consistency check on meta data level by comparing the repository with the operational data store.

A method of transforming and warehousing raw electronic data which is stored in a first application data model into a second relational data model and loading data into said second relational data model, said method comprising the steps of:

• (i) defining an idealized data model in a database server for at least one data source, said defining step comprising:
• importing metadata from said data source into a product;
• refining data model keys and relationships in said product if necessary;
• and
• improving and/or selecting one or more new table- and column-names to define said data source;
• and
• (ii) processing the idealized data model in a database server for at least one data source by converting the first application data model into the second relational data model, said processing step comprising:
• converting one or more original data source keys and relationships to a surrogate key-based model by creating at least one destination table with an idealized name format;
• creating a surrogate key conversion table for each destination table;
• and
• importing data through a parallel processing of said destination tables.