Currency reduction for predictive human resources synchronization rectification

A system for automatically detecting and rectifying data anomalies to synchronize datasets, the system comprising:

• one or more databases or distributed file systems communicating over an electronic network and configured for data storage;
• and
• a computing device comprising one or more processors, memory, programmable instructions or applications, machine learning algorithm modules calculation engines, and at least one operating system (OS), and configured to:
• receive, raw human resource data from an external source, the raw human resource data including compensation data and store the received raw human resource data in data storage according to a source data schema;
• configure the raw human resource data stored in data storage according to a source data schema including currency reduction on the compensation data and store the configured data in data storage;
• process the configured data to determine anomalies in in the compensation data using a machine learning predictive model and storing the results in data storage, wherein to process the configured data the computing device:
• segregates and prepares test datasets and training datasets;
• generates and evaluates feature sets comprising extracting relevant content from the transformed data;
• fits, validates, and tests predictive model comprising:
• training a logistic regression model applying machine learning algorithms to the at least one predictive model and calculating model accuracy wherein feature sets are tested in the at least one predictive model for accuracy with respect to results corresponding to the training datasets to create a hierarchy of one or more candidate models wherein the one or more candidate models that produce a highest level of accuracy using training data of the training datasets are selected for use in the at least one predictive model,
• wherein accuracy is calculated as a ratio of correctly predicted observation to total observations and wherein a currency model with a highest probability is used for a predicted currency of a row singular currency value is set as a correct value for the training dataset which is fed into logistic regression algorithms to output a number of features sets that are individually applied to the logistic regression algorithms using training data for both the test datasets and training datasets;
• and
• after logistic regression models comprising the at least one predictive model for all relevant currencies have been calculated, the at least one predictive model is fit with test data of the test dataset and the at least one predictive model is run to identify potential errant rows or records in the raw data and test data of the test dataset, wherein training data and test data are applied using the feature set with the highest level of accuracy to the logistic regression model that calculates the features and creates predictions for which currencies belong to which job records, where anomalous predicted currencies for the test data are identified as differentials between predicted currency of the test dataset and currency used in an economic zone and/or currency code, wherein if currency information is not present for both the economic zone and the currency code, a row is tagged as anomalous;
• deploys the predictive model and transforms currency data;
• and
• calculates second level metrics using model data;
• and
• a graphical user interface configured to generate and display the results of the predictive model indicating anomalies in the compensation data stored in the data storage together with potential reasons for anomalies and suggested correction to rectify and synchronize the data.

A non-transitory computer-readable medium for automatically detecting and rectifying data anomalies to synchronize datasets, the non-transitory computer-readable medium comprising stored electronic instructions that when executed on at least one computing device perform steps comprising:

• receiving, using a computing device, raw human resource data from an external source, the raw human resource data including compensation data and storing the received raw human resource data in data storage according to a source data schema;
• configuring, using a computing device, the raw human resource data stored in data storage according to a source data schema including currency reduction on the compensation data and storing the configured data in data storage;
• processing, using a computing device, the configured data to determine anomalies in in the compensation data using a machine learning predictive model and storing the results in data storage, wherein processing the configured data comprises:
• segregating and preparing test datasets and training datasets;
• generating and evaluating feature sets comprising extracting relevant content from the transformed data;
• fitting, validating, and testing predictive model comprising the steps of:
• training a logistic regression model applying machine learning algorithms to the at least one predictive model and calculating model accuracy wherein feature sets are tested in the at least one predictive model for accuracy with respect to results corresponding to the training datasets to create a hierarchy of one or more candidate models wherein the one or more candidate models that produce a highest level of accuracy using training data of the training datasets are selected for use in the at least one predictive model,
• wherein accuracy is calculated as a ratio of correctly predicted observation to total observations and wherein a currency model with a highest probability is used for a predicted currency of a row singular currency value is set as a correct value for the training dataset which is fed into logistic regression algorithms to output a number of features sets that are individually applied to the logistic regression algorithms using training data for both the test datasets and training datasets;
• and
• after logistic regression models comprising the at least one predictive model for all relevant currencies have been calculated, the at least one predictive model is fit with test data of the test dataset and the at least one predictive model is run to identify potential errant rows or records in the raw data and test data of the test dataset, wherein training data and test data are applied using a feature set with the highest level of accuracy to the logistic regression model that calculates the features and creates predictions for which currencies belong to which job records, where anomalous predicted currencies for the test data are identified as differentials between predicted currency of the test dataset and currency used in an economic zone and/or currency code, wherein if currency information is not present for both the economic zone and the currency code, a row is tagged as anomalous;
• deploying the predictive model and transforming currency data;
• and
• calculating second level metrics using model data;
• and
• generating and displaying, through a graphical user interface of a computing device, the results of the predictive model indicating anomalies in the compensation data stored in the data storage together with potential reasons for anomalies and suggested correction to rectify and synchronize the data.

A computer implemented method for automatically detecting and rectifying data anomalies to synchronize datasets, the method comprising:

• receiving, using a computing device, raw data from an external data source;
• performing data enrichment on the raw data received;
• performing deep character level inspection on enriched data;
• extracting numerical data from the enriched data;
• analyzing the numerical data and performing currency reduction;
• calculating first level metrics;
• transforming currency data and updating data records with transformed data;
• segregating and preparing test datasets and training datasets;
• generating and evaluating feature sets comprising extracting relevant content from the transformed data;
• fitting, validating and testing a predictive model comprising the steps of:
• training a logistic regression model applying machine learning algorithms to the at least one predictive model and calculating model accuracy wherein feature sets are tested in the at least one predictive model for accuracy with respect to results corresponding to the training datasets to create a hierarchy of one or more candidate models wherein the one or more candidate models that produce a highest level of accuracy using training data of the training datasets are selected for use in the at least one predictive model,
• wherein accuracy is calculated as a ratio of correctly predicted observation to total observations and wherein a currency model with a highest probability is used for a predicted currency of a row singular currency value is set as a correct value for the training dataset which is fed into logistic regression algorithms to output a number of features sets that are individually applied to the logistic regression algorithms using training data for both the test datasets and training datasets;
• and
• after logistic regression models comprising the at least one predictive model for all relevant currencies have been calculated, the at least one predictive model is fit with test data of the test dataset and the at least one predictive model is run to identify potential errant rows or records in the raw data and test data of the test dataset, wherein training data and test data are applied using a feature set with the highest level of accuracy to the logistic regression model that calculates the features and creates predictions for which currencies belong to which job records, where anomalous predicted currencies for the test data are identified as differentials between predicted currency of the test dataset and currency used in an economic zone and/or currency code, wherein if currency information is not present for both the economic zone and the currency code, a row is tagged as anomalous;
• deploying the predictive model and transforming currency data;
• calculating second level metrics using model data;
• and
• automatically generating and presenting, to a user, through a graphical user interface, results of the predictive model deployed in a displayable format for further end user remediation comprising indicating instances of erroneous data in the raw data together with a corresponding listing of potential reasons why the raw data is erroneous, and suggested corrections to rectify the raw data and synchronize the raw data.

A system for automatically detecting and rectifying data anomalies to synchronize datasets, the system comprising:

• one or more databases or distributed file systems communicating over an electronic network and configured for data storage,
• a computing device comprising one or more processors, memory, programmable instructions or applications, machine learning algorithm modules calculation engines, and at least one operating system (OS), and configured to:
• receive raw data from an external data source;
• perform data enrichment on the raw data received;
• perform deep character level inspection on enriched data;
• extract numerical data from the enriched data;
• analyze the numerical data and perform currency reduction;
• calculate first level metrics;
• transform currency data and updating data records with transformed data;
• segregate and prepare test datasets and training datasets;
• generate and evaluate feature sets comprising extracting relevant content from the transformed data;
• fit, validate and test a predictive model comprising the steps of:
• training a logistic regression model applying machine learning algorithms to the at least one predictive model and calculating model accuracy wherein feature sets are tested in the at least one predictive model for accuracy with respect to results corresponding to the training datasets to create a hierarchy of one or more candidate models wherein the one or more candidate models that produce a highest level of accuracy using training data of the training datasets are selected for use in the at least one predictive model,
• wherein accuracy is calculated as a ratio of correctly predicted observation to total observations and wherein a currency model with a highest probability is used for a predicted currency of a row singular currency value is set as a correct value for the training dataset which is fed into logistic regression algorithms to output a number of features sets that are individually applied to the logistic regression algorithms using training data for both the test datasets and training datasets;
• and
• after logistic regression models comprising the at least one predictive model for all relevant currencies have been calculated, the at least one predictive model is fit with test data of the test dataset and the at least one predictive model is run to identify potential errant rows or records in the raw data and test data of the test dataset, wherein training data and test data are applied using a feature set with the highest level of accuracy to the logistic regression model that calculates the features and creates predictions for which currencies belong to which job records, where anomalous predicted currencies for the test data are identified as differentials between predicted currency of the test dataset and currency used in an economic zone and/or currency code, wherein if currency information is not present for both the economic zone and the currency code, a row is tagged as anomalous;
• deploy the predictive model to transform currency data;
• calculate second level metrics using model data;
• and
• a graphical user interface configured to automatically generate and present, to a user, results of the predictive model deployed in a displayable format for further end user remediation comprising a list indicating instances of erroneous data in the raw data together with a corresponding listing of potential reasons why the raw data is erroneous, and suggested corrections to rectify the raw data and synchronize the raw data.

A non-transitory computer-readable medium for automatically detecting and rectifying data anomalies to synchronize datasets, the non-transitory computer-readable medium comprising stored electronic instructions that when executed on at least one computing device perform steps comprising:

• receiving, using the at least one computing device, raw data from a data source;
• performing data enrichment on the raw data received;
• performing deep character level inspection on enriched data;
• extracting numerical data from the enriched data;
• analyzing the numerical data and performing currency reduction;
• calculating first level metrics;
• transforming currency data and updating data records with transformed data;
• segregating and preparing test datasets and training datasets;
• generating and evaluating feature sets comprising extracting relevant content from the transformed data;
• fitting, validating and testing a predictive model comprising the steps of:
• training a logistic regression model applying machine learning algorithms to the at least one predictive model and calculating model accuracy wherein feature sets are tested in the at least one predictive model for accuracy with respect to results corresponding to the training datasets to create a hierarchy of one or more candidate models wherein the one or more candidate models that produce a highest level of accuracy using training data of the training datasets are selected for use in the at least one predictive model,
• wherein accuracy is calculated as a ratio of correctly predicted observation to total observations and wherein a currency model with a highest probability is used for a predicted currency of a row singular currency value is set as a correct value for the training dataset which is fed into logistic regression algorithms to output a number of features sets that are individually applied to the logistic regression algorithms using training data for both the test datasets and training datasets;
• and
• after logistic regression models comprising the at least one predictive model for all relevant currencies have been calculated, the at least one predictive model is fit with test data of the test dataset and the at least one predictive model is run to identify potential errant rows or records in the raw data and test data of the test dataset, wherein training data and test data are applied using a feature set with the highest level of accuracy to the logistic regression model that calculates the features and creates predictions for which currencies belong to which job records, where anomalous predicted currencies for the test data are identified as differentials between predicted currency of the test dataset and currency used in an economic zone and/or currency code, wherein if currency information is not present for both the economic zone and the currency code, a row is tagged as anomalous;
• deploying the predictive model and transforming currency data;
• calculating second level metrics using model data;
• and
• automatically generating and presenting, to a user, through a graphical user interface, results of the predictive model deployed in a displayable format for further end user remediation comprising a list indicating instances of erroneous data in the raw data together with a corresponding listing of potential reasons why the raw data is erroneous, and suggested corrections to rectify the raw data and synchronize the raw data.