Fast quantised training of trainable modules

A method for training a trainable module that maps one or more input variables onto one or more output variables through an internal processing chain, the internal processing chain being characterized by a set of parameters, the method comprising the following steps:

• providing at least one learning data set that includes learning values of the input variables and associated learning values of the output variables;
• providing a list of discrete values from which the parameters characterizing the internal processing chain are to be selected during the training, the discrete values being selected such that the discrete values can be stored with a specified number N of bits as a fixed-point number without loss of quality;
• mapping, by the trainable model, the learning values contained in the learning data set of the input variables onto assessment values of the output variables;
• evaluating a specified cost function that characterizes both a deviation of the assessment values of the output variables from the learning values, contained in the learning data set, of the output variables, and also a deviation of at least one of the parameters of the internal processing chain from at least one of the discrete values in the list;
• and
• adjusting at least one of the parameters of the internal processing chain with a goal of improving a value of the cost function, wherein a value range of the parameters, and/or of a gradient of the cost function, is limited during the adjustment of the at least one of the parameters, using the discrete values.

A method for producing a trainable module that maps one or more input variables onto one or more output variables through an internal processing chain, the internal processing chain being characterized by a set of parameters, the method comprising the following steps:

• ascertaining discrete values for the parameters of the internal processing chain through training by:
• providing at least one learning data set that includes learning values of the input variables and associated learning values of the output variables,
• providing a list of discrete values from which the parameters characterizing the internal processing chain are to be selected during the training, the discrete values being selected such that the discrete values can be stored with a specified number N of bits as a fixed-point number without loss of quality,
• mapping, by the trainable model, the learning values contained in the learning data set of the input variables onto assessment values of the output variables,
• evaluating a specified cost function that characterizes both a deviation of the assessment values of the output variables from the learning values, contained in the learning data set, of the output variables, and also a deviation of at least one of the parameters of the internal processing chain from at least one of the discrete values in the list, and
• adjusting at least one of the parameters of the internal processing chain with a goal of improving a value of the cost function, wherein a value range of the parameters, and/or of a gradient of the cost function, is limited during the adjustment of the at least one of the parameters, using the discrete values;
• wherein the cost function contains a weighted sum of at least:
• a first contribution that characterizes a deviation of the assessment values of the output variables from the learning values, contained in the learning data set, of the output variables, and
• a second contribution that characterizes a deviation of at least one parameter of the internal processing chain from at least one discrete value in the list,
• wherein, as the training progresses, a weighting of the first contribution is reduced and a weighting of the second contribution is increased,
• wherein the discrete values in the list have whole-number values, and discrete values that are numerically adjacent to one another differing in each case by a step width that is a power of two of a non-negative whole number;
• implementing the internal processing chain of the trainable module in an arithmetic unit that is configured for representation and/or processing of the parameters characterizing the internal processing chain in fixed-point arithmetic;
• setting the parameters, in the arithmetic unit, to the ascertained discrete values.

A method, comprising:

• training a trainable module that maps one or more input variables onto one or more output variables through an internal processing chain, the internal processing chain being characterized by a set of parameters, the training including:
• providing at least one learning data set that includes learning values of the input variables and associated learning values of the output variables,
• providing a list of discrete values from which the parameters characterizing the internal processing chain are to be selected during the training, the discrete values being selected such that the discrete values can be stored with a specified number N of bits as a fixed-point number without loss of quality,
• mapping, by the trainable model, the learning values contained in the learning data set of the input variables onto assessment values of the output variables,
• evaluating a specified cost function that characterizes both a deviation of the assessment values of the output variables from the learning values, contained in the learning data set, of the output variables, and also a deviation of at least one of the parameters of the internal processing chain from at least one of the discrete values in the list, and
• adjusting at least one of the parameters of the internal processing chain with a goal of improving a value of the cost function, wherein a value range of the parameters, and/or of a gradient of the cost function, is limited during the adjustment of the at least one of the parameters, using the discrete values;
• operating the trainable module by supplying one or more input variables to it;
• as a function of the output variables supplied by the trainable module, controlling a vehicle or a robot or a quality control system or a system for monitoring an area, based on sensor data.

A non-transitory machine-readable data carrier on which is stored a computer program for training a trainable module that maps one or more input variables onto one or more output variables through an internal processing chain, the internal processing chain being characterized by a set of parameters, the computer program, when executed by one or more computers, causing the one or more computers to perform the following steps:

• providing at least one learning data set that includes learning values of the input variables and associated learning values of the output variables;
• providing a list of discrete values from which the parameters characterizing the internal processing chain are to be selected during the training, the discrete values being selected such that the discrete values can be stored with a specified number N of bits as a fixed-point number without loss of quality;
• mapping, by the trainable model, the learning values contained in the learning data set of the input variables onto assessment values of the output variables;
• evaluating a specified cost function that characterizes both a deviation of the assessment values of the output variables from the learning values, contained in the learning data set, of the output variables, and also a deviation of at least one of the parameters of the internal processing chain from at least one of the discrete values in the list;
• and
• adjusting at least one of the parameters of the internal processing chain with a goal of improving a value of the cost function, wherein a value range of the parameters, and/or of a gradient of the cost function, is limited during the adjustment of the at least one of the parameters, using the discrete values.

A computer configured to train a trainable module that maps one or more input variables onto one or more output variables through an internal processing chain, the internal processing chain being characterized by a set of parameters, the computer configured to:

• provide at least one learning data set that includes learning values of the input variables and associated learning values of the output variables;
• provide a list of discrete values from which the parameters characterizing the internal processing chain are to be selected during the training, the discrete values being selected such that the discrete values can be stored with a specified number N of bits as a fixed-point number without loss of quality;
• map, by the trainable model, the learning values contained in the learning data set of the input variables onto assessment values of the output variables;
• evaluate a specified cost function that characterizes both a deviation of the assessment values of the output variables from the learning values, contained in the learning data set, of the output variables, and also a deviation of at least one of the parameters of the internal processing chain from at least one of the discrete values in the list;
• and
• adjust at least one of the parameters of the internal processing chain with a goal of improving a value of the cost function, wherein a value range of the parameters, and/or of a gradient of the cost function, is limited during the adjustment of the at least one of the parameters, using the discrete values.