Method and device for extracting an optimal network architecture for solving the target task

A method for extracting an optimal network architecture for solving a target task, the method comprising the following steps:

• providing a supermodel pre-trained based on labeled training data for solving the target task, wherein the supermodel includes a plurality of pre-trained operations, each of which can be assigned at least one architecture weight;
• adding at least one LoRA module to at least one respective one of the operations of the supermodel, wherein each of the at least one LoRA module includes respective trainable weights, wherein the at least one of the operations of the supermodel remains unchanged or is frozen during addition of the trainable weights;
• training the pre-trained supermodel by training the respective weights of each of the at least one LORA module and the at least one of the operations, until a certain training criterion is reached;
• extracting an optimal network architecture for solving the target task from the trained supermodel based on the trained architecture weights;
• and
• providing the extracted optimal network architecture for solving the target task.

A non-transitory computer-readable data carrier on which is stored a computer program for extracting an optimal network architecture for solving a target task, the computer program, when executed by a computer, causing the computer to perform the following steps:

• providing a supermodel pre-trained based on labeled training data for solving the target task, wherein the supermodel includes a plurality of pre-trained operations, each of which can be assigned at least one architecture weight;
• adding at least one LoRA module to at least one respective one of the operations of the supermodel, wherein each of the at least one LoRA module includes respective trainable weights, wherein the at least one of the operations of the supermodel remains unchanged or is frozen during addition of the trainable weights;
• training the pre-trained supermodel by training the respective weights of each of the at least one LoRA module and the at least one of the operations, until a certain training criterion is reached;
• extracting an optimal network architecture for solving the target task from the trained supermodel based on the trained architecture weights;
• and
• providing the extracted optimal network architecture for solving the target task.

A device configured to extract an optimal network architecture for solving a target task, wherein the device comprises an evaluation and computing unit that is configured to carry out the following steps:

• providing a supermodel pre-trained based on labeled training data for solving the target task, wherein the supermodel includes a plurality of pre-trained operations, each of which can be assigned at least one architecture weight;
• adding at least one LoRA module to at least one of the operations of the supermodel, wherein each of the at least one LoRA module includes respective trainable weights, wherein the at least one of the operations of the supermodel remains unchanged or is frozen during addition of the respective trainable weights;
• training the pre-trained supermodel by training the respective weights of the at least one LoRA module and the at least one of the operations until a certain training criterion is reached;
• extracting an optimal network architecture for solving the target task from the trained supermodel based on the trained architecture weights;
• and
• providing the extracted optimal network architecture for solving the target task.