Systems and methods for improving error tolerance in processing an input file

A computer-based method for improving error tolerance in processing a plurality of input files, said method implemented on a computer system comprising a gateway processor, a cluster database, a plurality of node-cluster nodes, a warehouse node, and a warehouse database, said method comprising:

• storing, in the warehouse database, a plurality of destination records, each of the destination records comprising data for a plurality of destination fields;
• receiving, at the gateway processor from an originator, each of the input files;
• receiving, by a load balancer from the gateway processor, the input files;
• distributing, by the load balancer, portions of each received input file to one of the plurality of node-cluster nodes;
• executing, by each node-cluster node of the plurality of node-cluster nodes, an inbound process and a core process, wherein the plurality of node-cluster nodes are programmed to execute the inbound process and the core process in parallel to each other node-cluster node, wherein the inbound process of each node-cluster node is programmed to perform steps including:
• receiving respective portions of the input files routed through the load balancer;
• extracting input data records from the input file, each of the input data records comprising data for a plurality of input fields, wherein the plurality of input fields correspond to the plurality of destination fields;
• and
• storing the input data records in the cluster database;
• executing, by the warehouse node, a matching engine programmed to access the stored input data records in the cluster database and, for each stored input data record, perform steps including:
• comparing the data in the plurality of input fields to the data in the destination fields of each of the destination records;
• generating a match score for each of the destination records based on the comparison of the data;
• generating a list of candidate destination records by ranking the match score for each destination record;
• transmitting the list of candidate destination records and the corresponding match scores to the cluster database, wherein the stored input data record is updated to include the list of candidate destination records and the corresponding match scores;
• wherein the core process of each node-cluster node of the plurality of node-cluster nodes is programmed to perform steps including:
• comparing, for each of the candidate destination records, a format of the data in at least one of the input fields to a mask associated with the corresponding destination field;
• adjusting the match score of the candidate destination record based on the comparison to the mask;
• generating an output record including the data from the destination fields of the candidate destination record having the highest adjusted match score;
• and
• transmitting the output record to the gateway processor.

At least one non-transitory computer-readable storage media having embodied thereon a first set of computer-executable instructions, a second set of computer-executable instructions, a third set of computer-executable instructions, and a fourth set of computer-executable instructions, wherein:

• when executed by at least one gateway processor, the first set of computer-executable instructions cause the gateway processor to receive, from an originator, a plurality of input files;
• when executed by a load balancer, the fourth set of computer-executable instructions cause the load balancer to receive the input files from the gateway processor and distribute portions of each received input file to one of a plurality of node-cluster nodes;
• when executed by each node-cluster node of the plurality of node-cluster nodes in communication with a cluster database, the second set of computer-executable instructions cause each node-cluster node to execute an inbound process programmed to:
• receive respective portions of the input files routed through the load balancer;
• extract input data records from the input file, each of the input data records comprising data for a plurality of input fields;
• and
• store the input data records in said cluster database;
• when executed by a warehouse node in communication with the cluster database and a warehouse database, the warehouse database storing a plurality of destination records, each of the destination records comprising data for a plurality of destination fields, wherein the plurality of destination fields correspond to the plurality of input fields, the third set of computer-executable instructions cause the warehouse node to call a matching engine programmed to access the stored input data records in the cluster database and, for each stored input data record:
• compare the data in the plurality of input fields to the data in the destination fields of each of the destination records;
• generate a match score for each of the destination records based on the comparison of the data;
• generate a list of candidate destination records by ranking the match score for each destination record;
• and
• transmit the list of candidate destination records and the corresponding match scores to the cluster database, wherein the stored input data record is updated to include the list of candidate destination records and the corresponding match scores;
• and
• when executed by each node-cluster node, the second set of computer-executable instructions further cause each node-cluster node to execute a core process programmed to, for each updated stored input data record:
• compare, for each of the candidate destination records, a format of the data in at least one of the input fields to a mask associated with the corresponding destination field;
• adjust the match score of the candidate destination record based on the comparison to the mask;
• generate an output record including the data from the destination fields of the candidate destination record having the highest adjusted match score;
• and
• transmit the output record to the gateway processor,
• wherein the plurality of node-cluster nodes are programmed to execute the inbound process and the core process in parallel to each other node-cluster node.