# US 2021/0279580 A1 · Machine learning system and a method, a computer program and a device for creating the machine learning system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279580 A1
- Document ID: US20210279580A1
- Status: Published application
- Kind: A1
- Application number: 17/260,065
- Filed: 2019-08-13
- Published: 2021-09-09
- Earliest claimed: 2018-09-26
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Thomas Pfeil
- Web page: https://patentgrep.com/patent/US20210279580A1

## CPC

- G06N 3/048: Activation functions
- G06N 3/08: Learning methods
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A machine learning system, in particular a deep neural network. The machine learning system includes a plurality of layers that are connected to one another. The layers each ascertain an output variable as a function of an input variable and at least one parameter that is stored in a memory. The parameters of those layers that are connected to a further, in particular preceding, layer are each stored in the memory using a higher resolution than the parameters of those layers that are connected to a plurality of further, in particular preceding, layers. In addition, A method, a computer program, and a device for creating the machine learning system, are described.

## Claims

### Claim at position 1

1-13. (canceled)

### Claim 14

A machine learning system, comprising:
a plurality of layers that are each connected at an input side to at least one further layer of the plurality of layers, each of the layers being configured to ascertain an output variable as a function of activations, each respective layer of the layers further configured to ascertain the activations as a function of at least one input variable of the respective layer and as a function of parameters that are assigned to the respective layer;
wherein a resolution of the parameters or of the activations is a function of whether or not one of the layers is connected immediately at the input side to a plurality of preceding layers.

### Claim 15 (depends on 14)

The machine learning system as recited in claim 14, wherein the machine learning system is a deep neural network.

### Claim 16 (depends on 14)

The machine learning system as recited in claim 14, wherein a resolution of the parameters or of the activations of a layer of the layers that is connected immediately at the input side to the plurality of preceding, layers is lower than a resolution of the parameters or of the activations of those layers that are immediately connected at the input side to exactly one layer.

### Claim 17 (depends on 14)

The machine learning system as recited in claim 14, wherein the parameters or activations of those layers that are connected immediately at the input side to exactly one layer each have a variable from a first list including a first specifiable number of quantized variables, and the parameters or activations of that layer that is immediately connected at the input side to a plurality of layers each have a variable from a second list including a second specifiable number of quantized variables, a quantization error of the variables of the first list being smaller than the quantization error of the variables of the second list.

### Claim 18 (depends on 14)

The machine learning system as recited in claim 14, wherein an input variable of an input layer of the machine learning system is a function of a sensor variable of a sensor, and an output layer of the machine learning system is connected to a control unit, the control unit ascertaining, as a function of the output variable of the output layer connected to the control unit, a control variable for controlling a physical actuator.

### Claim 19 (depends on 18)

The machine learning system as recited in claim 18, wherein an at least partly autonomous robot and/or a production system and/or a personal assistant and/or an access system and/or a monitoring system and/or a medical imaging system, is controlled as a function of the ascertained control variable.

### Claim 20

A method for creating a compressed machine learning system from a provided machine learning system, comprising the following steps:
providing the machine learning system having a plurality of layers each having an input side; and
selecting a resolution of parameters or activations of the compressed machine learning system by reducing a corresponding resolution of parameters or activations of the provided machine learning system, the resolution of the parameters or of the activations of the compressed machine learning system being selected as a function of whether or not one of the layers of the provided machine learning system is connected immediately at the input side to a multitude of the layers.

### Claim 21 (depends on 20)

The method as recited in claim 20, wherein the corresponding resolution of the parameters or of the activations of a layer the plurality of layers of the provided machine learning system that is connected immediately at the input side to the multitude of layers is more strongly reduced than a corresponding resolution of the parameters or of the activations of those layers of the provided machine learning system that are connected immediately at the input side to exactly one layer.

### Claim 22 (depends on 21)

The method as recited in claim 21, wherein the reduction of the resolution is carried out by reducing a number of bits for representing the variables.

### Claim 23 (depends on 22)

The method as recited in claim 22, wherein the reduction of the number of bits is carried out as a function of a number of quantization steps.

### Claim 24 (depends on 20)

The method as recited in claim 20, further comprising:
providing training data that respectively include input variables and output variables assigned to the input variables; and
training the provided machine learning system, the parameters being adapted during the training in such a way that, as a respective function of the input variables of the training data, the machine learning system respectively ascertains their assigned output variable.

### Claim 25

A non-transitory machine-readable storage medium on which is stored a computer program for creating a compressed machine learning system from a provided machine learning system, the computer program, when executed by a computer, causing the computer to perform the following steps:
providing the machine learning system having a plurality of layers each having an input side; and
selecting a resolution of parameters or activations of the compressed machine learning system by reducing a corresponding resolution of parameters or activations of the provided machine learning system, the resolution of the parameters or of the activations of the compressed machine learning system being selected as a function of whether or not one of the layers of the provided machine learning system is connected immediately at the input side to a multitude of the layers.

### Claim 26

A device configured to create a compressed machine learning system from a provided machine learning system, the device configured to:
provide the machine learning system having a plurality of layers each having an input side; and
select a resolution of parameters or activations of the compressed machine learning system by reducing a corresponding resolution of parameters or activations of the provided machine learning system, the resolution of the parameters or of the activations of the compressed machine learning system being selected as a function of whether or not one of the layers of the provided machine learning system is connected immediately at the input side to a multitude of the layers.
