Systems and methods for machine learning based rule discovery for data migration

A method for deriving classification rules from documents and a database using rule-based machine learning, the method comprising:

• extracting, by a server computing device, a first plurality of variables from documents corresponding to an organization;
• extracting, by the server computing device, a second plurality of variables from a database corresponding to the organization;
• filtering, by the server computing device, the extracted second plurality of variables based on at least one of null values, repeat variables, location variables, ID variables, or date variables;
• deriving, by the server computing device, a first plurality of classification rules based on the first plurality of variables using a rule-based machine learning algorithm, comprising:
• a) executing the rule-based machine learning algorithm using the first plurality of variables as input to derive a classification rule and identify a subset of the first plurality of variables that satisfy the classification rule,
• b) removing the identified subset of variables from the first plurality of variables,
• c) repeating steps a) and b) using the remaining first plurality of variables as input to identify additional subsets of the remaining variables that satisfy additional classification rules, and
• d) storing the classification rules derived by the rule-based machine learning algorithm in step a) as the first plurality of classification rules;
• calculating, by the server computing device, an accuracy of the derived first plurality of classification rules;
• deriving, by the server computing device, a second plurality of classification rules based on the first plurality of variables and filtered second plurality of variables using the rule-based machine learning algorithm, comprising:
• e) executing the rule-based machine learning algorithm using the first plurality of variables and one of the filtered second plurality of variables as input to derive a classification rule and identify a subset of the first plurality of variables and one of the filtered second plurality of variables that satisfy the classification rule,
• f) removing the identified subset of variables from the first plurality of variables and the one of the filtered second plurality of variables,
• g) updating the calculated accuracy of the derived first plurality of classification rules based upon the first plurality of variables and the one of the filtered second plurality of variables,
• h) repeating steps e), f) and g) using the remaining first plurality of variables and another one of the filtered second plurality of variables as input to identify additional subsets of the remaining variables that satisfy additional classification rules, and
• i) storing the classification rules generated by the rule-based machine learning algorithm in step e) as the second plurality of classification rules;
• and
• identifying, by the server computing device, a suggested additional variable from the filtered second plurality of variables for inclusion in the first plurality of variables based upon the updated accuracy of the derived first plurality of classification rules;
• and
• generating, by the server computing device, for display the derived first plurality of classification rules, the derived second plurality of classification rules, the updated accuracy of the derived first plurality of classification rules, and the suggested additional variable.