Method and apparatus for generating machine learning model by using distributed computing framework

An apparatus for generating a machine learning model, the apparatus comprising:

• a generation unit, configured to generate a plurality of model parameter combinations, wherein in the plurality of model parameter combinations, values of model parameters in a model parameter combination are different from values of model parameters in another model parameter combination;
• and generate machine learning models respectively based on the plurality of model parameter combinations, each machine learning model being generated based on a model parameter combination, wherein the model parameters indicate an associated relationship between input vectors and output vectors of the machine learning models;
• a division unit, configured to execute a dividing operation:
• dividing preset machine learning data into training data and validation data;
• a processing unit, configured to execute training and validation operations:
• training the machine learning models in parallel respectively based on the training data, wherein each of the machine learning models is trained with the training data;
• validating learning accuracies of the trained machine learning models in parallel respectively based on the validation data to obtain validation scores, wherein each validation score indicates a ratio of consistency between a data type corresponding to an output vectors output based on a piece of validation data by each machine leaning model and a type of the piece of validation data;
• and
• an execution unit, configured to execute a model generation operation:
• determining an optimal model parameter combination based on a machine learning model to be generated based on the validation scores, and generating a machine learning model corresponding to the optimal model parameter combination.

An apparatus, comprising:

• a processor;
• and
• a memory,
• wherein the memory stores computer readable instructions capable being executed by the processor, the processor executes a method for generating a machine learning model when the computer readable instructions are executed, and the method comprises:
• generating a plurality of model parameter combinations, wherein in the plurality of model parameter combinations, values of model parameters in a model parameter combination are different from values of model parameters in another model parameter combination;
• and generating machine learning models respectively based on the plurality of model parameter combinations, each machine learning model being generated based on a model parameter combination, wherein the model parameters indicate an associated relationship between input vectors and output vectors of the machine learning models;
• executing a dividing operation:
• dividing preset machine learning data into training data and validation data;
• executing training and validation operations:
• training the machine learning models in parallel respectively based on the training data, wherein each of the machine learning models is trained with the training data;
• validating learning accuracies of the trained machine learning models in parallel respectively based on the validation data to obtain validation scores, wherein each validation scores indicates a ratio of consistency between a data type corresponding to an output vector output based on a piece of validation data by each machine leaning model and a type of the piece of validation data;
• and
• executing a model generation operation:
• determining an optimal model parameter combination corresponding to a machine learning model to be generated based on the validation scores, and generating a machine learning model based on the optimal model parameter combination.