Method and system for generating best performing data models for datasets in a computing environment

A system for generating best performing data models for datasets in a computing environment, the system comprises:

• one or more hardware processors;
• and
• a memory coupled to the one or more hardware processors, wherein the memory comprises a plurality of subsystems in the form of programmable instructions executable by the one or more hardware processors, wherein the plurality of subsystems comprises:
• a dataset receiver module configured for obtaining an input dataset from one or more external data sources, wherein the input dataset comprises data variables and data samples;
• a dataset segmentation module configured for dividing the obtained dataset into at least three parts based on a type of the data variables and the data samples, wherein each part of the dataset comprises samples from the input the dataset, and wherein a first part of the dataset comprises training data, a second part of the dataset comprises testing data, and a third part of the dataset comprises validation data;
• a data model method retrieval module configured for retrieving one or more data modeling methods from a pool of data modeling methods stored in a database, wherein the one or more data modeling methods comprises one or more parameter ranges;
• a data model generator module configured for generating a first set of data models for the first part of the dataset by using the retrieved one or more data modeling methods, wherein input to generate the first set of data models is the first part of the dataset;
• a fitness function determination module configured for determining a first fitness function for the first part of the dataset based on the generated first set of data models;
• the data model generator module further configured for generating a second set of data models for the second part of the dataset by validating the first set of data models and the determined first fitness function with the second part of the dataset;
• the fitness function determination module configured for determining a second fitness function associated with the second part of the dataset based on the generated second set of data models;
• a best performing data model determination module configured for determining at least one best performing data model among the first set of data models and the second set of data models by comparing the determined first fitness function with the determined second fitness function;
• and
• an output module configured for outputting the determined at least one best performing data model on a user interface of a user device.

A non-transitory computer-readable storage medium having instructions stored therein that, when executed by a hardware processor, causes the processor to perform method steps comprising:

• obtaining an input dataset from one or more external data sources, wherein the input dataset comprises data variables and data samples;
• dividing the obtained dataset into at least three parts based on a type of the data variables and the data samples, wherein each part of the dataset comprises an samples from the input datasets, and wherein the first part of the dataset comprises training data, the second part of the dataset comprises testing data, and third part of the dataset comprises validation data;
• retrieving one or more data modelling methods from a pool of data modelling methods stored in a database, wherein the one or more data modelling methods comprises one or more parameter ranges;
• generating a first set of data models for the first part of the dataset by using the retrieved one or more data modelling methods, wherein input to generate the first set of data models is the first part of the dataset;
• determining a first fitness function for the first part of the dataset based on the generated first set of data models;
• generating the second set of data models for the second part of the dataset by validating the first set of data models and the determined first fitness function with the second part of the dataset;
• determining a second fitness function associated with the second part of the dataset based on the generated second set of data models;
• determining at least one best performing data model among the first set of data models and the second set of data models by comparing the determined first fitness function with the determined second fitness function;
• and
• outputting the determined at least one best performing data model on a user interface of a user device.