# US 10,311,075 B2 · Refactoring of databases to include soft type information > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,075 B2 - Document ID: US10311075B2 - Status: Granted - Kind: B2 - Application number: 14/562,138 - Filed: 2014-12-05 - Published: 2015-06-18 - Granted: 2019-06-04 - Earliest claimed: 2013-12-13 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Nicholas Kanellos; Jonathan Limburn; Ivan M. Milman; Martin Oberhofer - Web page: https://patentgrep.com/patent/US10311075B2 ## CPC - G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Method and system are provided for refactoring of databases to include soft type information. The method may include: carrying out data profiling of soft type data to generate an attribute list with mapping suggestions to a hardened database structure; generating a data model definition and extract, transform, and load logic for transforming the soft type data based on the attribute list and mapping suggestions; executing the data model definition to create a new or modified database structure; moving data from an existing database structure to the new or modified database structure; and regenerating a services interface for access to the data. ## Claims ### Claim 1 A computer-implemented method for refactoring a database, comprising: profiling soft type data to generate an attribute list and one or more mapping suggestions for mapping the soft type data to another database structure, wherein the soft type data include data stored in a database table external of a defined database table to extend the defined database table to include additional attributes without modifying a definition of the defined database table, wherein the soft type data in the external database table includes one or more tuples each corresponding to an additional attribute and comprising a value for the additional attribute and a row identifier indicating a row in the defined database table associated with the additional attribute, and wherein profiling the soft type data to generate an attribute list and one or more mapping suggestions to another database structure includes: performing column analysis on the soft type data and applying statistics on frequency, uniqueness and value distribution of attributes to determine additional columns present in the soft type data extending the defined database table; generating a data model definition and extract, transform, and load logic for transforming the soft type data based on the attribute list and at least one of the one or more mapping suggestions; executing the data model definition to create a target database structure incorporating the additional columns as structured columns into one or more defined database tables, wherein the additional columns are directly accessible by a query in a relational structured query language specifying the additional columns and received by the database from an external application; moving data from an existing database structure to the target database structure, wherein moving data from the existing database structure to the target database structure includes: processing requests by one or more external applications for data records by performing an access operation to access the requested data records from soft type data within the external database table; and moving data records of soft type data incrementally from the external database table to the additional structured columns of the one or more defined database tables of the target database structure as each data record is requested, wherein each data record is transformed from the soft type data to hard type data and stored in the target database structure after the access operation for that data record; and providing access to the soft type data within the defined database tables of the target database structure. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving user inputs of soft type information and configured thresholds for new attributes of a hardened form to be generated for the soft type information. ### Claim 3 (depends on 2) The method of claim 2, wherein the configured thresholds are a selected one of relative thresholds and absolute thresholds for the new attributes of the hardened form. ### Claim 4 (depends on 1) The method of claim 1, wherein executing the data model definition further creates a new database structure with a foreign key relationship to a corresponding existing database structure with one or more structured fields containing the soft type data. ### Claim 5 (depends on 1) The method of claim 1, wherein the target database structure includes a modified database structure with at least one structured field added to contain the soft type data. ### Claim 6 (depends on 1) The method of claim 1, wherein the one or more mapping suggestions comprises a new type of database structure with a foreign key relationship to the existing database structure with at least one structured field containing the soft type data. ### Claim 7 (depends on 1) The method of claim 1, wherein the one or more mapping suggestions to another database structure comprises modifying a database structure to add one or more structured fields containing the soft type data. ### Claim 8 (depends on 1) The method of claim 1, further comprising providing a user input for reviewing and modifying the generated attribute list and one or more mapping suggestions. ### Claim 9 (depends on 1) The method of claim 1, further including providing an intercepting and rerouting mechanism for read and write requests to move the data from the existing database structure to the target database structure. ### Claim 10 (depends on 1) The method of claim 1, further including providing feedback to a user on a suggested mapping in the form of a color indicating a level of confidence that the mapping meets user requirements. ### Claim 11 A system for refactoring a database, comprising: at least one hardware processor configured to: profile soft type data to generate an attribute list and one or more mapping suggestions for mapping the soft type data to another database structure, wherein the soft type data include data stored in a database table external of a defined database table to extend the defined database table to include additional attributes without modifying a definition of the defined database table, wherein the soft type data in the external database table includes one or more tuples each corresponding to an additional attribute and comprising a value for the additional attribute and a row identifier indicating a row in the defined database table associated with the additional attribute, and wherein profiling the soft type data to generate an attribute list and one or more mapping suggestions to another database structure includes: performing column analysis on the soft type data and applying statistics on frequency, uniqueness and value distribution of attributes to determine additional columns present in the soft type data extending the defined database table; generate a data model definition and extract, transform, and load logic for transforming the soft type data based on the attribute list and at least one of the one or more mapping suggestions; execute the data model definition to create a target database structure incorporating the additional columns as structured columns into one or more defined database tables, wherein the additional columns are directly accessible by a query in a relational structured query language specifying the additional columns and received by the database from an external application; move data from an existing database structure to the target database structure, wherein moving data from the existing database structure to the target database structure includes: processing requests by one or more external applications for data records by performing an access operation to access the requested data records from soft type data within the external database table; and moving data records of soft type data incrementally from the external database table to the additional structured columns of the one or more defined database tables of the target database structure as each data record is requested, wherein each data record is transformed from the soft type data to hard type data and stored in the target database structure after the access operation for that data record; and provide access to the soft type data within the defined database tables of the target database structure. ### Claim 12 (depends on 11) The system of claim 11, wherein the at least one hardware processor is further configured to: receive user inputs of soft type information and configured thresholds for new attributes of a hardened form to be generated for the soft type information. ### Claim 13 (depends on 12) The system of claim 12, wherein the configured thresholds are a selected one of relative thresholds and absolute thresholds for the new attributes of the hardened form. ### Claim 14 (depends on 11) The system of claim 11, wherein executing the data model definition further creates a new database structure with a foreign key relationship to a corresponding existing database structure with one or more structured fields containing the soft type data. ### Claim 15 (depends on 11) The system of claim 11, wherein the target database structure includes a modified database structure with at least one structured field added to contain the soft type data. ### Claim 16 (depends on 11) The system of claim 11, wherein the one or more mapping suggestions comprises a new type of database structure with a foreign key relationship to the existing database structure with at least one structured field containing the soft type data. ### Claim 17 (depends on 11) The system of claim 11, wherein the one or more mapping suggestions to another database structure comprises modifying a database structure to add one or more structured fields containing the soft type data. ### Claim 18 (depends on 11) The system of claim 11, wherein the at least one hardware processor is further configured to: provide a user input for reviewing and modifying the generated attribute list and one or more mapping suggestions. ### Claim 19 A computer program product for refactoring a database, comprising: a non-transitory computer readable storage medium having computer readable program code embodied therewith for execution on a processing system, the computer readable program code comprising computer readable program code configured to: profile soft type data to generate an attribute list and one or more mapping suggestions for mapping the soft type data to another database structure, wherein the soft type data include data stored in a database table external of a defined database table to extend the defined database table to include additional attributes without modifying a definition of the defined database table, wherein the soft type data in the external database table includes one or more tuples each corresponding to an additional attribute and comprising a value for the additional attribute and a row identifier indicating a row in the defined database table associated with the additional attribute, and wherein profiling the soft type data to generate an attribute list and one or more mapping suggestions to another database structure includes: performing column analysis on the soft type data and applying statistics on frequency, uniqueness and value distribution of attributes to determine additional columns present in the soft type data extending the defined database table; generate a data model definition and extract, transform, and load logic for transforming the soft type data based on the attribute list and at least one of the one or more mapping suggestions; execute the data model definition to create a target database structure incorporating the additional columns as structured columns into one or more defined database tables, wherein the additional columns are directly accessible by a query in a relational structured query language specifying the additional columns and received by the database from an external application; move data from an existing database structure to the target database structure, wherein moving data from the existing database structure to the target database structure includes: processing requests by one or more external applications for data records by performing an access operation to access the requested data records from soft type data within the external database table; and moving data records of soft type data incrementally from the external database table to the additional structured columns of the one or more defined database tables of the target database structure as each data record is requested, wherein each data record is transformed from the soft type data to hard type data and stored in the target database structure after the access operation for that data record; and provide access to the soft type data within the defined database tables of the target database structure. ### Claim 20 (depends on 19) The computer program product of claim 19, wherein the computer readable program code is further configured to: receive user inputs of soft type information and configured thresholds for new attributes of a hardened form to be generated for the soft type information. ### Claim 21 (depends on 20) The computer program product of claim 20, wherein the configured thresholds are a selected one of relative thresholds and absolute thresholds for the new attributes of the hardened form. ### Claim 22 (depends on 19) The computer program product claim 19, wherein executing the data model definition further creates a new database structure with a foreign key relationship to a corresponding existing database structure with one or more structured fields containing the soft type data. ### Claim 23 (depends on 19) The computer program product of claim 19, wherein the one or more mapping suggestions comprises a new type of database structure with a foreign key relationship to the existing database structure with at least one structured field containing the soft type data. ### Claim 24 (depends on 19) The computer program product of claim 19, wherein the one or more mapping suggestions to another database structure comprises modifying a database structure to add one or more structured fields containing the soft type data. ### Claim 25 (depends on 19) The computer program product of claim 19, wherein the computer readable program code is further configured to: provide a user input for reviewing and modifying the generated attribute list and one or more mapping suggestions.