Method and system of creating balanced dataset

A system of creating an output dataset comprising:

• one or more processors in a data processing device communicably connected to a memory, wherein the memory stores a plurality of processor-executable instructions which upon execution cause the one or more processors to:
• receive a dataset comprising a plurality of input data files,
• wherein each input data file comprises attribute values corresponding to a presence of a plurality of attributes;
• create a bucket dataset comprising the input data file selected based on a highest first selection value, wherein the first selection value is a quantification value, wherein the quantification value for each of the input data files is determined based on a probability of occurrence of each of the attributes in the corresponding input data file;
• iteratively sample the dataset until all input data files of the dataset are added into the bucket list, wherein the iterative sampling for each iteration comprises:
• create a subset dataset including subset data files, wherein subset data files are determined based on a summation data file, wherein the summation data file is determined based on summation of attribute values for each of the attributes for each of the input data file of the bucket dataset;
• add the summation data file to the summation dataset;
• determine a second selection value for each of the subset data files of the subset dataset, wherein the second selection value is a quantification value of each the subset data files determined based on probability of occurrence of each of the attributes in each of the corresponding subset data file;
• and
• add to the bucket dataset the input data file of the updated dataset corresponding to the subset data file with highest second selection value, wherein the dataset is updated by decrementing the input data file added to the bucket dataset;
• determine a third selection value for each of the summation data files of the summation dataset;
• and
• determine the output dataset as the bucket dataset determined for the sampling iteration based on an output criterion, wherein the output criterion is based on the third selection value.

A method of creating an output dataset, the method comprising:

• receiving, by one or more processors of a computing device, a dataset from a plurality of data sources, wherein the dataset comprises a plurality of input data files;
• wherein each input data file from the plurality of input data files comprises one or more pre-defined attributes;
• iteratively sampling the dataset based on a pre-defined type of sampling;
• and
• determining the output dataset based on the pre-defined type of sampling and an output criterion associated to the pre-defined type of sampling, wherein the output dataset comprises a threshold number of input data files and a threshold value of distribution of the input data files for each of the pre-defined attributes.

A non-transitory computer-readable medium storing computer-executable instructions for creating an output dataset, the computer-executable instructions configured for:

• receiving a dataset comprising a plurality of input data files,
• wherein each input data file comprises attribute values corresponding to a presence of a plurality of attributes;
• creating a bucket dataset comprising the input data file selected based on a highest first selection value, wherein the first selection value is a quantification value, wherein the quantification value for each of the input data files is determined based on a probability of occurrence of each of the attributes in the corresponding input data file;
• iteratively sampling the dataset until all input data files of the dataset are added into the bucket list, wherein the iterative sampling for each iteration comprises:
• creating a subset dataset including subset data files, wherein subset data files are determined based on a summation data file, wherein the summation data file is determined based on summation of attribute values for each of the attributes for each of the input data file of the bucket dataset;
• adding the summation data file to the summation dataset;
• determining a second selection value for each of the subset data files of the subset dataset, wherein the second selection value is a quantification value of each the subset data files determined based on probability of occurrence of each of the attributes in each of the corresponding subset data file;
• and
• adding to the bucket dataset the input data file of the updated dataset corresponding to the subset data file with highest second selection value, wherein the dataset is updated by decrementing the input data file added to the bucket dataset;
• determining a third selection value for each of the summation data files of the summation dataset;
• and
• determining the output dataset as the bucket dataset determined for the sampling iteration based on an output criterion, wherein the output criterion is based on the third selection value.