Metafaas architecture for training on serverless instances

A system for meta learning using distributed training, the system comprising:

• a memory storing instructions;
• one or more Input/Output (I/O) interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to:
• initialize a base model to be trained using a serverless architecture;
• spawn a plurality of serverless instances of the serverless architecture and sharing a copy of the base model with each of the plurality of serverless instances;
• and
• generate a trained model via an iterative training process using a gradient-based meta learning approach applied on the copy of the base model via the plurality of serverless instances, the iterative training process comprising:
• generating and adding to a task pool, a plurality meta batches each comprising a plurality of tasks, wherein the plurality of tasks in each of the plurality of meta batches is received at irregular time intervals;
• randomly sampling a meta batch from amongst the plurality of meta batches from the task pool;
• uniformly distributing and loading the plurality of tasks from the sampled meta batch across the plurality of serverless instances;
• generating a local model by each of the plurality of serverless instances by training the copy of the base model using a corresponding task loaded on each of the plurality of serverless instances and updating one or more task specific parameters for the copy of the base model at a current level of the iterative training process;
• computing a plurality of gradients corresponding to each of the local model from the updated one or more task specific parameters at the current level of the iterative training process, wherein the plurality of gradients is stored in a cloud storage-based communication channel;
• aggregating the plurality of gradients corresponding to the local model;
• updating the base model in accordance with the aggregated plurality of gradients;
• and
• sharing a copy of the updated base model with each of the plurality of serverless instances for successive level of the iterative training process, wherein the iterative training process terminates once the base model converges to provide the trained model.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• initializing a base model to be trained using a serverless architecture;
• spawning a plurality of serverless instances of the serverless architecture and sharing a copy of the base model with each of the plurality of serverless instances;
• and
• generating a trained model via an iterative training process using a gradient-based meta learning approach applied on the copy of the base model via the plurality of serverless instances, the iterative training process comprising:
• generating and adding to a task pool, a plurality meta batches each comprising a plurality of tasks, wherein the plurality of tasks in each of the plurality of meta batches is received at irregular time intervals;
• randomly sampling a meta batch from amongst the plurality of meta batches from the task pool;
• uniformly distributing and loading the plurality of tasks from the sampled meta batch across the plurality of serverless instances;
• generating a local model by each of the plurality of serverless instances by training the copy of the base model using a corresponding task loaded on each of the plurality of serverless instances and updating one or more task specific parameters for the copy of the base model at a current level of the iterative training process;
• computing a plurality of gradients corresponding to each of the local model from the updated one or more task specific parameters at the current level of the iterative training process, wherein the plurality of gradients is stored in a cloud storage-based communication channel;
• aggregating the plurality of gradients corresponding to the local model;
• updating the base model in accordance with the aggregated plurality of gradients;
• and
• sharing a copy of the updated base model with each of the plurality of serverless instances for successive level of the iterative training process, wherein the iterative training process terminates once the base model converges to provide the trained model.