System and method for generating synthetic data records

A system for generating synthetic data records in a distributed computing architecture, the system comprising:

• a memory storing program instructions;
• at least one processor configured to execute program instructions stored in the memory;
• and a synthetic data production engine executed by the at least one processor, and configured to:
• retrieve a predefined dataset from an external resource, the predefined dataset comprising data fields specific to a selected operating field of synthetic data records, wherein the dataset is validated during retrieval including format validation, data completeness, and exclusion of special characters;
• extract relevant data values associated with selected relevant data fields from a predefined dataset including data fields within the validated dataset, the selection of relevant field is based on the operating field and user defined inputs, wherein the extracted data values are stored in a configurable format;
• define dynamic rules for generating synthetic data values of one or more specific data fields out of the selected relevant data fields based on an analysis of the extracted data values or user-inputs or number of combination thereof, wherein the rules are customised based on specific scenarios including handling missing data values associated with the specific data fields such that the rules ensure that the generated data conform to predefined standards;
• evaluate a number of possible synthetic data records, wherein combinations between the extracted data values associated with two or more of the selected relevant data fields are computed, the computed combinations is the number of possible data records, and wherein the combinations between data values of two or more selected relevant data fields are computed by deriving a cartesian product of the highest and lowest data values associated with corresponding two or more of the selected relevant fields;
• and
• generate the evaluated number of synthetic data records based on the extracted data values and the defined rules, wherein the synthetic data records are formatted according to the predefined standards for the selected operating field and include both the extracted and newly generated data values, ensuring compatibility with existing data processing systems.

A computer program product comprising:

• a non-transitory computer-readable medium having computer-readable program code stored thereon, the computer-readable program code comprising instructions that, when executed by a processor, cause the processor to:
• retrieve a predefined dataset from an external resource, the predefined dataset comprising data fields specific to a selected operating field of synthetic data records, wherein the dataset is validated during retrieval including format validation, data completeness, and exclusion of special characters;
• extract relevant data values associated with selected relevant data fields within the validated dataset, the selection of relevant field is based on the operating field and user defined inputs, wherein the extracted data values are stored in a configurable format;
• define dynamic rules for generating synthetic data values of one or more specific data fields out of the selected relevant data fields based on an analysis of the extracted data values or user-inputs or number of combination thereof, wherein the rules are customised based on specific scenarios including handling missing data values associated with the specific data fields such that the rules ensure that the generated data conform to predefined standards;
• evaluate a number of possible synthetic data records, wherein combinations between the extracted data values associated with two or more of the selected relevant data fields are computed, the computed combinations is the number of possible data records, and wherein the combinations between data values of two or more selected relevant data fields are computed by deriving a cartesian product of the highest and lowest data values associated with corresponding two or more of the selected relevant fields;
• and
• generate the evaluated number of synthetic data records based on the extracted data values and the defined rules, wherein the synthetic data records are formatted according to the predefined standards for the selected operating field and include both the extracted and newly generated data values, ensuring compatibility with existing data processing systems.