Method for performing federated learning based on moe and lora, electronic device supporting the same, and storage medium

A method of performing federated learning by an electronic device, the method comprising:

• identifying a plurality of expert models corresponding to a plurality of external electronic devices configured to perform federated learning, wherein the plurality of expert models are included in a large language model (LLM), and wherein the LLM includes a gating network pre-trained to identify at least one expert model, corresponding to input data, among the plurality of expert models, and
• transmitting, to the plurality of external electronic devices, through the communication circuitry of the electronic device, the gating network and each of the plurality of expert models, for training the gating network and the each of the plurality of expert models.

A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by at least one processor individually or collectively, cause an electronic device to:

• identify a plurality of expert models corresponding to a plurality of external electronic devices configured to perform federated learning, wherein the plurality of expert models are included in a large language model (LLM), and wherein the LLM includes a gating network pre-trained to identify at least one expert model, corresponding to input data, among the plurality of expert models, and
• transmit, to the plurality of external electronic devices, through the communication circuitry, the gating network and each of the plurality of expert models, for training the gating network and the each of the plurality of expert models.