System and method of training heterogenous models using stacked ensembles on decentralized data

A method of training a stacked ensemble, the method comprising:

• training a first sub-model with a first local data set accessible locally at a first node of a decentralized system, obfuscating the trained first sub-model by compiling the first sub-model using a runtime application to define a common set of operators and file format, freezing the compiled first sub-model to be non-trainable, and employing software licensing to ensure proper access eligibility to the compiled first sub-model;
• adding the first obfuscated sub-model to a blockchain network;
• receiving the first obfuscated sub-model at a second node of the decentralized system, wherein the second node comprises a second obfuscated sub-model trained with a second local data set;
• constructing, at the second node, in response to a consensus agreement for an architecture selected by a contract-executed vote on a distributed ledger, a local instance of the stacked ensemble, the stacked ensemble comprising the first obfuscated sub-model, the second obfuscated sub-model, and at least one combinator layer of the first node comprising a trainable parametric layer;
• training, at the second node, the at least one trainable parametric layer using the second local data set comprising training data not accessible to the first node;
• adding an obfuscated second sub-model generated from the local instance of the stacked ensemble to the blockchain network after the training;
• receiving, at the first node, the obfuscated second sub-model generated from the local instance of the stacked ensemble from the blockchain network;
• constructing, using the combinator layer, a second local instance of the stacked ensemble comprising outputs of the first obfuscated sub-model and the second obfuscated sub-model from the blockchain network, wherein the trainable parametric layer's weights are updated during a backpropagation phase of training, an input to the combinator layer of the first node being a N-Dimensional matrix formed by concatenation of outputs of the first obfuscated sub-model and the second obfuscated sub-model;
• and
• generating, using a machine learning framework, an inference based on the second local instance of the stacked ensemble with a different machine learning model architecture than the second node.

A non-transitory machine-readable storage medium comprising instructions executable by a processor, the instructions programming the processor to:

• train a first sub-model with a first local data set accessible locally at a first node of a decentralized system, wherein the first node implements a first machine learning model architecture, obfuscate the trained first sub-model by compiling the first sub-model using a runtime application to define a common set of operators and file format, freezing the compiled first sub-model to be non-trainable, and employing software licensing to ensure proper access eligibility to the compiled first sub-model;
• add the first obfuscated sub-model to a blockchain network;
• receive the first obfuscated sub-model at a second node of the decentralized system, wherein the second node implements a second machine learning model architecture that is different than the first machine learning model architecture, and wherein the second node comprises a second obfuscated sub-model trained with a second local data set;
• construct, at the second node, in response to a consensus agreement for an architecture selected by a contract-executed vote on a distributed ledger, a local instance of the stacked ensemble, the stacked ensemble comprising the first obfuscated sub-model, the second obfuscated sub-model, and at least one combinator layer of the first node comprising a trainable parametric layer;
• train, at the second node, the at least one trainable parametric layer using the second local data set comprising training data not accessible to the first node;
• add an obfuscated second sub-model generated from the local instance of the stacked ensemble to the blockchain network after the training;
• receive, at the first node, the obfuscated second sub-model generated from the local instance of the stacked ensemble from the blockchain network;
• construct, using the combinator layer, a second local instance of the stacked ensemble comprising outputs of the first obfuscated sub-model and the second obfuscated sub-model from the blockchain network, wherein the trainable parametric layer's weights are updated during a backpropagation phase of training, an input to the combinator layer of the first node being a N-Dimensional matrix formed by concatenation of outputs of the first obfuscated sub-model and the second obfuscated sub-model;
• and
• generate, using a machine learning framework, an inference based on the second local instance of the stacked ensemble with a different machine learning model architecture than the second node.