# US 2021/0342453 A1 · Private model utility by minimizing expected loss under noise

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342453 A1
- Document ID: US20210342453A1
- Status: Published application
- Kind: A1
- Application number: 16/862,056
- Filed: 2020-04-29
- Published: 2021-11-04
- Earliest claimed: 2020-04-29
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Klas LEINO; Jorge GUAJARDO MERCHAN
- Web page: https://patentgrep.com/patent/US20210342453A1

## CPC

- G06F 21/60: Protecting data
- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N 3/045: Combinations of networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Training of a model is performed to minimize expected loss under noise (ELUN) while maintaining differential privacy. Noise is added to weights of a machine learning model as random samples drawn from a noise distribution, the noise being added in accordance with a privacy budget. The ELUN is minimized by using a loss function that anticipates noise added to the weights of the machine learning model, to find a point in the parameter space for which loss is robust to the noise in the weights. The addition of noise and the minimization of the ELUN are iterated until the weights converge and optimization constraints are satisfied. The model is utilized on arbitrary inputs while protecting the privacy of training data used to train the model.

## Claims

### Claim 1

A method for training and utilizing a model to minimize expected loss under noise (ELUN) while maintaining differential privacy, the method comprising:
adding noise to weights of a machine learning model as random samples drawn from a noise distribution, the noise being added in accordance with a privacy budget;
minimizing the ELUN by using a loss function that anticipates noise added to the weights of the machine learning model, to find a point in the parameter space for which loss is robust to the noise in the weights;
iterating the addition of noise and the minimization of the ELUN until the weights converge and optimization constraints are satisfied; and
utilizing the model on arbitrary inputs while protecting the privacy of training data used to train the model.

### Claim 2 (depends on 1)

The method of claim 1, wherein the noise includes Laplacian noise

### Claim 3 (depends on 1)

The method of claim 1, wherein the noise includes Gaussian noise.

### Claim 4 (depends on 1)

The method of claim 1, wherein the noise is approximated via random samples drawn from the Laplace distribution.

### Claim 5 (depends on 1)

The method of claim 1, wherein the machine learning model is a linear machine learning model.

### Claim 6 (depends on 1)

The method of claim 1, wherein the machine learning model includes one or more of a Support Vector Machine, a Convolutional Neural Network (CNN), or a Deep Neural Network (DNN).

### Claim 7 (depends on 1)

The method of claim 1, wherein minimizing the ELUN includes optimization according to gradient descent.

### Claim 8 (depends on 1)

The method of claim 1, wherein the ELUN is denoted by , and given by:
ℒ
~
⁡
(
θ
,
(
x
,
y
)
)
=
𝔼
θ
′
∼
P
θ
⁡
[
ℒ
⁡
(
θ
′
,
(
x
,
y
)
)
]
where is a loss function defined over model parameters, θ, and labeled points, (x,y); and Pc is a noise distribution over possible model parameters, centered at c.

### Claim 9

A system for training and utilizing a model to minimize expected loss under noise (ELUN) while maintaining differential privacy, the system comprising:
a memory storing a machine learning model; and
a computing device, programmed to
add noise to weights of the machine learning model as random samples drawn from a noise distribution, the noise being added in accordance with a privacy budget;
minimize the ELUN by using a loss function that anticipates noise added to the weights of the machine learning model, to find a point in the parameter space for which loss is robust to the noise in the weights;
iterate the addition of noise and the minimization of the ELUN until the weights converge and optimization constraints are satisfied; and
utilize the model on arbitrary inputs while protecting the privacy of training data used to train the model.

### Claim 10 (depends on 9)

The system of claim 9, wherein the noise includes one or more of Laplacian noise or Gaussian noise.

### Claim 11 (depends on 9)

The system of claim 9, wherein the noise is approximated via random samples drawn from the Laplace distribution.

### Claim 12 (depends on 9)

The system of claim 9, wherein the machine learning model includes one or more of a linear machine learning model, a Support Vector Machine, a Convolutional Neural Network (CNN), or a Deep Neural Network (DNN).

### Claim 13 (depends on 9)

The system of claim 9, wherein minimizing the ELUN includes optimization according to gradient descent.

### Claim 14 (depends on 9)

The system of claim 9, wherein the ELUN is denoted by , and given by:
ℒ
~
⁡
(
θ
,
(
x
,
y
)
)
=
𝔼
θ
′
∼
P
θ
⁡
[
ℒ
⁡
(
θ
′
,
(
x
,
y
)
)
]
where is a loss function defined over model parameters, θ, and labeled points, (x,y); and Pc is a noise distribution over possible model parameters, centered at c.

### Claim 15

A non-transitory computer readable medium comprising instructions for training and utilizing a model to minimize expected loss under noise (ELUN) while maintaining differential privacy that, when executed by a processor, cause the processor to perform operations including to:
add noise to weights of a machine learning model as random samples drawn from a noise distribution, the noise being added in accordance with a privacy budget;
minimize the ELUN by using a loss function that anticipates noise added to the weights of the machine learning model, to find a point in the parameter space for which loss is robust to the noise in the weights;
iterate the addition of noise and the minimization of the ELUN until the weights converge and optimization constraints are satisfied; and
utilize the model on arbitrary inputs while protecting the privacy of training data used to train the model.

### Claim 16 (depends on 15)

The medium of claim 15, wherein the noise includes one or more of Laplacian noise or Gaussian noise.

### Claim 17 (depends on 15)

The medium of claim 15, wherein the noise is approximated via random samples drawn from the Laplace distribution.

### Claim 18 (depends on 15)

The medium of claim 15, wherein the machine learning model is a linear machine learning model.

### Claim 19 (depends on 15)

The medium of claim 15, wherein minimizing the ELUN includes optimization according to gradient descent.

### Claim 20 (depends on 15)

The medium of claim 15, wherein the ELUN is denoted by , and given by:
ℒ
~
⁡
(
θ
,
(
x
,
y
)
)
=
𝔼
θ
′
∼
P
θ
⁡
[
ℒ
⁡
(
θ
′
,
(
x
,
y
)
)
]
where is a loss function defined over model parameters, θ, and labeled points, (x,y); and Pc is a noise distribution over possible model parameters, centered at c.
