System and method for real-time monitoring of data replication for high availability

A method, comprising:

• electronically receiving a plurality of messages from a first external device, wherein each of the plurality of messages includes target data and record data, and the target data indicates where the record data should be stored;
• storing target data for each of the received plurality of messages in a message log table, wherein the message log table records target data for each of the plurality of messages;
• processing each of the received plurality of messages by:
• processing a selected message using a selected one of a current plurality of jobs by analyzing the target data included in the selected message to determine a plurality of target databases to store the record data associated with the selected message;
• electronically storing the associated record data in the determined plurality of target databases;
• removing the selected message from the message log table when the selected one of the current plurality of jobs completes processing the selected message;
• and
• updating a message processing rate in a job status table for the selected one of the current plurality of jobs, wherein the job status table stores job data on each of a current plurality of jobs, and the job data includes at least the message processing rate for each of the current plurality of jobs;
• and
• adjusting the current plurality of jobs while processing each of the received plurality of messages by:
• determining a total number of messages that the current plurality of jobs is able to process in a predetermined period of time based on at least a processing rate for each of the current plurality of jobs;
• determining a current number of pending messages by analyzing the target data in the message log table;
• determining an efficiency parameter by subtracting the total number of messages from the current number of pending messages;
• starting an additional job when the efficiency parameter is less than a first predetermined threshold;
• and
• adding the additional job to the current plurality of jobs.

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:

• electronically receive a plurality of messages from a first external device, wherein each of the plurality of messages includes target data and record data, and the target data indicates where the record data should be stored;
• store target data for each of the received plurality of messages in a message log table, wherein the message log table records target data for each of the plurality of messages;
• process each of the received plurality of messages by:
• processing a selected message using a selected one of a current plurality of jobs by analyzing the target data included in the selected message to determine a plurality of target databases to store the record data associated with the selected message;
• electronically storing the associated record data in the determined plurality of target databases;
• removing the selected message from the message log table when the selected one of the current plurality of jobs completes processing the selected message;
• and
• updating a message processing rate in a job status table for the selected one of the current plurality of jobs, wherein the job status table stores job data on each of a current plurality of jobs, and the job data includes at least the message processing rate for each of the current plurality of jobs;
• and
• adjust the current plurality of jobs while processing each of the received plurality of messages by:
• determining a total number of messages that the current plurality of jobs is able to process in a predetermined period of time based on at least a processing rate for each of the current plurality of jobs;
• determining a current number of pending messages by analyzing the target data in the message log table;
• determining an efficiency parameter by subtracting the total number of messages from the current number of pending messages;
• starting an additional job when the efficiency parameter is less than a first predetermined threshold;
• and
• adding the additional job to the current plurality of jobs.