Device and computer implemented method for adapting a model to tasks

A device for adapting a model to tasks, the device comprising:

• at least one processor;) and
• at least one non-transitory memory, wherein the at least one non-transitory memory includes instructions that are executable by the at least one processor, and that, when executed by the at least one processor cause the device to execute a method for adapting the model to tasks, the method including the following steps:
• providing the model, wherein the model includes a linear layer configured to map a multidimensional input of the layer depending on weights to a multidimensional output of the layer, experts, and a router gate configured to adapt the model to different tasks,
• providing the input to the router gate,
• determining an output of the experts depending on an output of the router gate in response to the input,
• modifying the model depending on the output of the experts,
• mapping the input with the modified layer to the output of the model,
• training a first expert of the experts with a first training method depending on the output of the model,
• training a second expert of the experts with a second training method depending on the output of the model,
• wherein the weights and the second expert are maintained unchanged in the training with the first training method, and the weights and the first expert are maintained unchanged in the training with the second training method.

A non-transitory computer-readable medium on which is stored a computer program including instructions for adapting a model to tasks, the instructions, when executed by a computer, causing the computer to perform the following steps comprising:

• providing the model, wherein the model includes a linear layer configured to map a multidimensional input of the layer depending on weights to a multidimensional output of the layer, experts, and a router gate configured to adapt the model to different tasks;
• providing the input to the router gate;
• determining an output of the experts depending on an output of the router gate in response to the input;
• modifying the model depending on the output of the experts;
• mapping the input with the modified layer to the output of the model;
• training a first expert of the experts with a first training method depending on the output of the model;
• training a second expert of the experts with a second training method depending on the output of the model;
• wherein the weights and the second expert are maintained unchanged in the training with the first training method, and the weights and the first expert are maintained unchanged in the training with the second training method.

A computer implemented data structure for adapting a model to tasks, the data structure comprising:

• at least one data field for the model, wherein the model includes a linear layer for mapping a multidimensional input of the layer depending on weights to a multidimensional output of the layer, wherein the model includes experts and a router gate for adapting the model to different tasks;
• at least one data field for input to the router gate;
• at least one data field for an output of the experts determined depending on an output of the router gate in response to the input to the router gate;
• at least one data filed for a modified layer determined by modifying the model depending on the output of the experts;
• at least one data filed for training a first expert of the experts with a first training method depending on the output of the model;
• and
• at least one data filed for training a second expert of the experts with a second training method depending on the output of the model, and maintaining the weights and the second expert unchanged in the training with the first training method, and maintaining the weights and the first expert unchanged in the training with the second training method.