Device and method for training a normalizing flow using self-normalized gradients

A training system configured to train a normalizing flow, wherein the normalizing flow is configured to determine a first output signal characterizing a likelihood or a log-likelihood of an input signal, wherein the normalizing flow includes at least one first layer, wherein the first layer includes trainable parameters and a layer input to the first layer is based on the input signal and the first output signal is based on a layer output of the first layer, the training system configured to:

• determine at least one training input signal;
• determine a training output signal for each of the at least one training input signal using the normalizing flow;
• determine a first loss value, wherein the first loss value is based on a likelihood or a log-likelihood of the at least one determined training output signal with respect to a predefined probability distribution;
• determine an approximation of a gradient of the trainable parameters of the first layer with respect to the first loss value, wherein the gradient is dependent on an inverse of a matrix of the trainable parameters and determining the approximation of the gradient is achieved by optimizing an approximation of the inverse;
• and
• update the trainable parameters of the first layer based on the approximation of the gradient.

A non-transitory machine-readable storage medium on which is stored a computer program for training a normalizing flow, wherein the normalizing flow is configured to determine a first output signal characterizing a likelihood or a log-likelihood of an input signal, wherein the normalizing flow includes at least one first layer, wherein the first layer includes trainable parameters and a layer input to the first layer is based on the input signal and the first output signal is based on a layer output of the first layer, the computer program, when executed by a computer, causing the computer to perform the following steps:

• determining at least one training input signal;
• determining a training output signal for each of the at least one training input signal using the normalizing flow;
• determining a first loss value, wherein the first loss value is based on a likelihood or a log-likelihood of the at least one determined training output signal with respect to a predefined probability distribution;
• determining an approximation of a gradient of the trainable parameters of the first layer with respect to the first loss value, wherein the gradient is dependent on an inverse of a matrix of the trainable parameters and determining the approximation of the gradient is achieved by optimizing an approximation of the inverse;
• and
• updating the trainable parameters of the first layer based on the approximation of the gradient.