Semantic discovery and mapping between data sources

A computer-implemented method for automatically generating a mapping suitable for analyzing data in a first data source and a second data source, and for identifying semantics and relationships between the first data source and the second data source, the method comprising:

• determining a binding condition between portions of data in the first data source and the second data source;
• using the binding condition to determine correlations between portions of data in the first data source and the second data source;
• and
• using the binding condition and the correlations to generate a mapping between portions of data in the first data source and the second data source;
• wherein generating the mapping between a plurality of columns of the first data source and the second data source uses a transformation function, and the method further comprises:
• identifying a join condition between a plurality of tables of the first data source and the second data source;
• constructing source data objects based on a set of relationships between the plurality of tables of the first data source and the second data source and an identified join condition;
• determining all possible transformation functions between the plurality of columns of the first data source and the plurality of columns of the second data source;
• calculating a hit rate as a number of rows that matches the value in the second data source after applying the transformation function to the values in the first data source;
• and
• identifying a best mapping corresponding to the transformation function between the plurality of columns of the first data source and the plurality of columns of the second data source based on the highest number hit rate.

A system for automatically generating a mapping suitable for analyzing data in a first data source and a second data source, and for identifying semantics and relationships between the first data source and the second data source, the system comprising:

• a computer system including at least one processor configured to:
• compare data between the first data source and the second data source;
• determine a binding condition between portions of data in the first data source and the second data source based upon the comparison, wherein the binding condition identifies data within the first and second data sources that map to each other;
• use the binding condition to determine correlations between portions of data in the first data source and the second data source, wherein the correlations identify data in the first data source with values that correspond to values of data in the second data source;
• and
• use the binding condition and the correlations to generate a mapping between portions of data in the first data source and the second data source, wherein the mapping includes a transformation function to generate data in the second data source from data in the first data source.

A system for automatically generating a mapping suitable for analyzing data in a first data source and a second data source, and for identifying semantics and relationships between the first data source and the second data source, the system comprising:

• a computer system including at least one processor configured to:
• determine a binding condition between portions of data in the first data source and the second data source;
• use the binding condition to determine correlations between portions of data in the first data source and the second data source;
• and
• use the binding condition and the correlations to generate a mapping between portions of data in the first data source and the second data source;
• wherein generating the mapping between a plurality of columns of the first data source and the second data source uses a transformation function, and the processor is further configured to:
• identify a join condition between a plurality of tables of the first data source and the second data source;
• construct source data objects based on a set of relationships between the plurality of tables of the first data source and the second data source and an identified join condition;
• determine all possible transformation functions between the plurality of columns of the first data source and the plurality of columns of the second data source;
• calculate a hit rate as a number of rows that matches the value in the second data source after applying the transformation function to the values in the first data source;
• and
• identify a best mapping corresponding to the transformation function between the plurality of columns of the first data source and the plurality of columns of the second data source based on the highest number hit rate.

A computer program product for automatically generating a mapping suitable for analyzing data in a first data source and a second data source, and for identifying semantics and relationships between the first data source and the second data source, the computer program product comprising:

• a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising computer readable program code configured to:
• compare data between the first data source and the second data source;
• determine a binding condition between portions of data in the first data source and the second data source based upon the comparison, wherein the binding condition identifies data within the first and second data sources that map to each other;
• use the binding condition to determine correlations between portions of data in the first data source and the second data source, wherein the correlations identify data in the first data source with values that correspond to values of data in the second data source;
• and
• use the binding condition and the correlations to generate a mapping between portions of data in the first data source and the second data source, wherein the mapping includes a transformation function to generate data in the second data source from data in the first data source.

A computer program product for automatically generating a mapping suitable for analyzing data in a first data source and a second data source, and for identifying semantics and relationships between the first data source and the second data source, the computer program product comprising:

• a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising computer readable program code configured to:
• determine a binding condition between portions of data in the first data source and the second data source;
• use the binding condition to determine correlations between portions of data in the first data source and the second data source;
• and
• use the binding condition and the correlations to generate a mapping between portions of data in the first data source and the second data source;
• wherein generating the mapping between a plurality of columns of the first data source and the second data source uses a transformation function, and the computer readable program code further comprising computer readable program code configured to:
• identify a join condition between a plurality of tables of the first data source and the second data source;
• construct source data objects based on a set of relationships between the plurality of tables of the first data source and the second data source and an identified join condition;
• determine all possible transformation functions between the plurality of columns of the first data source and the plurality of columns of the second data source;
• calculate a hit rate as a number of rows that matches the value in the second data source after applying the transformation function to the values in the first data source;
• and
• identify a best mapping corresponding to the transformation function between the plurality of columns of the first data source and the plurality of columns of the second data source based on the highest number hit rate.