Device, data structure and method for tuning weights of a neural network of a model

A device for tuning weights of a neural network of a model for processing input of the model, the input including or representing information about a technical system, and outputting an output of the model for operating a technical system, the device comprising:

• at least one processor;
• and
• at least one non-transitory memory, wherein the at least one non-transitory memory includes instructions that are executable by the at least one processor, and that, when executed by the at least one processor, cause the device to execute the method for tuning the weights wherein the model including a linear layer for mapping a multidimensional input of the layer depending on the weights to a multidimensional output of the layer, wherein the model is configured to determine the input of the layer depending on the input of the model, wherein the model is configured to determine the output of the model depending on the output of the layer, wherein the method includes the following steps:
• providing training data including the input of the model and a ground truth for the output of the model corresponding to the input of the model in the training data,
• providing a set of tuning methods for tuning the weights,
• determining principal components decomposition of a weight matrix including the weights, wherein the principal component decomposition includes a matrix formed by eigenvectors of a covariance matrix of the weight matrix,
• determining eigenvalues of the covariance matrix corresponding to the eigenvectors,
• rearranging the eigenvectors in the matrix in an order resulting in a monotonically decreasing order of the eigenvalues that are associated with the eigenvectors,
• rearranging the weights in the weight matrix according to the order in that the eigenvectors are rearranged in the matrix,
• partitioning the weight matrix into groups of weights,
• associating at least one group of the groups with a method of tuning selected from the set of tuning methods, and
• tuning the weights in the at least one group on the training data with the tuning method, leaving at least one group of the groups unaltered in the tuning.

A non-transitory computer-readable medium on which is stored a computer program including instructions for tuning weights of a neural network of a model for processing input of the model, the input including or representing information about a technical system, and outputting an output of the model for operating a technical system, the model including a linear layer for mapping a multidimensional input of the layer depending on the weights to a multidimensional output of the layer, wherein the model is configured to determine the input of the layer depending on the input of the model, wherein the model is configured to determine the output of the model depending on the output of the layer, wherein the instructions, when executed by a computer, causing the computer to perform the following steps:

• providing training data including the input of the model and a ground truth for the output of the model corresponding to the input of the model in the training data;
• providing a set of tuning methods for tuning the weights;
• determining principal components decomposition of a weight matrix including the weights, wherein the principal component decomposition includes a matrix formed by eigenvectors of a covariance matrix of the weight matrix;
• determining eigenvalues of the covariance matrix corresponding to the eigenvectors;
• rearranging the eigenvectors in the matrix in an order resulting in a monotonically decreasing order of the eigenvalues that are associated with the eigenvectors;
• rearranging the weights in the weight matrix according to the order in that the eigenvectors are rearranged in the matrix;
• partitioning the weight matrix into groups of weights;
• associating at least one group of the groups with a method of tuning selected from the set of tuning methods;
• and
• tuning the weights in the at least one group on the training data with the tuning method, leaving at least one group of the groups unaltered in the tuning.

A computer implemented data structure for tuning weights of a neural network of a model for processing input of the model, the input including or representing information about a technical system, and outputting an output of the model for operating a technical system, the data structure comprising:

• at least one data field for the model, wherein the model includes a linear layer for mapping a multidimensional input of the layer depending on the weights to a multidimensional output of the layer, wherein the model is configured to determine the input of the layer depending on the input of the model, wherein the model is configured to determine the output of the model depending on the output of the layer;
• at least one data field for training data including the input of the model and a ground truth for the output of the model corresponding to the input of the model in the training data;
• at least one data field for a set of tuning methods for tuning the weights;
• at least one data field for principal components decomposition of a weight matrix including the weights, wherein the principal component decomposition includes a matrix formed by eigenvectors of a covariance matrix of the weight matrix;
• at least one data field for eigenvalues of the covariance matrix corresponding to the eigenvectors;
• and
• at least one data field for associating at least one group of weights of the weight matrix with a method of tuning selected from the set of tuning methods.