# US 2023/0421371 A1 · Method and device for preserving privacy of linear regression distributed learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0421371 A1
- Document ID: US20230421371A1
- Status: Published application
- Kind: A1
- Application number: 18/251,335
- Filed: 2021-11-02
- Published: 2023-12-28
- Earliest claimed: 2020-10-30
- Assignees: Inesc Tec—Instituto De Engenharia De Sistemas E Computadores, Tecnologia E CiêNcia
- Applicants: Inesc Tec - Instituto De Engenharia De Sistemas E Computadores, Tecnologia E CiêNcia
- Inventors: Ricardo Jorge GOMES DE SOUSA BENTO BESSA; Carla Sofia DA SILVA GONÇALVES
- Web page: https://patentgrep.com/patent/US20230421371A1

## CPC

- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- H04L 9/0869: {involving random numbers or seeds}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- H04L 9/0825: {using asymmetric-key encryption or public key infrastructure [PKI], e.g. key signature or public key certificates}
- H04L 2209/46: Secure multiparty computation, e.g. millionaire problem
- Y04S 40/20: Information technology specific aspects, e.g. CAD, simulation, modelling, system security
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- Y04S: SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS

## Abstract

The present disclosure relates to a method and device for preserving privacy of linear regression distributed learning, in particular for preserving privacy of linear regression distributed learning using a LASSO (least absolute shrinkage and selection operator)-VAR (vector autoregressive) model, further in particular for preserving privacy of distributed learning using a LASSO-VAR model using convex optimization like alternating direction method of multipliers (ADMM) or using coordinate descent optimization. It is disclosed a device and method for preserving privacy of a linear regression model used in distributed learning by a set of agents indirectly sharing covariate data or target data when sharing coefficient matrixes for said model, comprising obtaining a perturbation matrix for encrypting the data being shared.

## Claims

### Claim 1

A computer implemented method for preserving privacy of a linear regression model used in distributed learning by a set of agents sharing covariate data and/or target data for said model, comprising obtaining an invertible random perturbation matrix, as an algebraic encryption key given by multiplication of a plurality of invertible randomly generated perturbation sub-matrixes, there being a perturbation sub-matrix for each respective agent,
wherein the invertible random perturbation matrix is to be used as an algebraic encryption key by left-multiplication of the data to be encrypted,
wherein an invertible perturbation sub-matrix is privately and randomly generated by each individual agent,
comprising the steps of:
for each individual agent, the individual agent privately generating a first random invertible matrix and a second random invertible matrix, multiplying a horizontally concatenated matrix of the data to be encrypted by said individual agent and said first matrix, with the second matrix, and sharing the resulting multiplication with a receiving agent selected from one of the remaining agents;
the receiving agent multiplying the perturbation sub-matrix of the receiving agent by the received multiplication, and sharing the resulting multiplication with another receiving agent selected from one of the remaining agents, repeating until a last remaining agent receives the multiplication, and then the last remaining agent multiplying the perturbation sub-matrix of the last remaining agent by the received multiplication, and sharing the resulting multiplication with the individual agent; and
for each individual agent, the individual agent recovering the encrypted data of said individual agent by computing the multiplication the shared resulting multiplication with the individual agent with the inverse of the second matrix, horizontally decatenating from the computed multiplication a left-side matrix and a right-side matrix, wherein the right-side matrix is the encrypted data of said individual agent as encrypted by the perturbation matrix;
for each individual agent, the individual agent sharing the encrypted data of said individual agent with all other agents.

### Claim 2

A computer implemented method for preserving privacy of a linear regression model used in distributed learning by a set of n agents sharing covariate data and/or target data for said model, according to the previous claim comprising obtaining an invertible random perturbation matrix M, as an algebraic encryption key M∈T×T,
given by multiplication of sub-matrixes MAi∈T×T given by:
M=MA1 . . . MAn (11)
such that MX is given by:
MX
=
[
M
A
1
⁢
…
⁢
M
A
n
⁢
X
A
1
︸
=
MX
A
1
,
…
,
M
A
1
⁢
…
⁢
M
A
n
⁢
X
A
n
]
︸
=
MX
A
n
,
(
12
)
where n is the number of agents, i is each agent, T is the number of observations, X∈T×ns is the data to be encrypted of all agents, X=[XA1, . . . , XAn], XAi∈T×s is the data to be encrypted of each i-th agent, s is the number of covariates observed from each agent, and wherein invertible matrix MAi is privately and randomly generated by each i-th agent,
comprising the steps of:
agent i-th privately generating random matrices CAi∈T×(r−s), DAi∈r×r, and sharing WAi∈T×r with a n-th agent, where WAi is obtained by:
WAi[XAi,CAi]DAi (13)
the n-th agent receiving WAi and sharing MAnWAi with a (n−1)-th agent, which then receives MAnWAi and shares MAn−1MAnWAi with a (n−2)-th agent; and repeating until a 1-st agent receives MA2 . . . MAnWAi and computes MWAi=MA1MA2 . . . MAnWAi;
the 1-st agent sends MWAi to agent i-th;
agent i-th recovers MXAi from:
[MCAi,MXAi]=MWAiDAi−1 (14)
and subsequently each agent sending the recovered MXAi as the encrypted data to be transmitted to the other agents.

### Claim 3 (depends on 2)

The computer implemented method according to claim 2, wherein said method steps are repeated for encrypting M−1, the inverse of the perturbation matrix M.

### Claim 4 (depends on 2)

The computer implemented method according to claim 2, further comprising obtaining a perturbation matrix Q∈ns×ns, for post-multiplying the data to be encrypted X=[XA1, . . . , XAn], where XAi∈T×s is the data to be encrypted of each i-th agent, as an algebraic encryption key Q, such that XQ is given by:
[
X
A
1
,
…
,
X
A
n
]
︸
=
X
⁢
[
Q
A
1
0
⋱
0
Q
A
n
]
︸
=
Q
=
[
X
A
1
⁢
Q
A
1
,
…
,
X
A
n
⁢
Q
A
n
]
.
(
10
)
where Q is a diagonal matrix formed by diagonal matrices QAi∈s×s which are random matrices privately generated by each agent i, the method comprising the steps of:
each i-th agent generating random matrix QAi,
and sharing XAiQAi with every other agent.

### Claim 5 (depends on 2)

The computer implemented method according to claim 2, wherein privacy of linear regression distributed learning is by using a LASSO (least absolute shrinkage and selection operator)-VAR (vector autoregressive) model.

### Claim 6 (depends on 5)

The computer implemented method according to claim 5, wherein the LASSO-VAR model uses convex optimization.

### Claim 7 (depends on 6)

The computer implemented method according to claim 6, wherein Z=[ZA1, . . . , ZAn] is a covariate matrix and Y=[ . . . , YAn] is a target matrix, where Z and Y are data to be encrypted and transmitted, where covariate matrix Z is split by model features, comprising each of said agents computing MZAi, MYAi and ZAiTM−1 for obtaining the encrypted data to be transmitted to the other agents.

### Claim 8 (depends on 4)

The computer implemented method according to claim 4, wherein ZQ is a covariate matrix and Y is a target matrix, where Z=[ZA1 . . . , ZAn] and Y=[YA1, . . . , YAn] are data to be encrypted and transmitted, where covariate matrix ZQ is split by model features, comprising each of said agents computing MZAi QAi, MYAi and QAiTZAiTM−1 for obtaining the encrypted data to be transmitted to the other agents.

### Claim 9 (depends on 8)

The computer implemented method according to claim 8, wherein ZAiQAi∈T×s has u unique values to recover and YAi∈T×g has v unique unknown values that are not in ZAi, comprising the steps of computing MZAiQAi and QAiTZAiTM−1 using the smaller integer r such that √{square root over (Ts−u)}<r<T/2∧r>p, and computing MYAi using the smaller integer r′ such that √{square root over (−u+Ts−r2−v+Tg)}<r′<T−2r∧r′>g.

### Claim 10 (depends on 8)

The computer implemented method for obtaining a non-encrypted LASSO-VAR model, according to claim 8, further comprising each agent:
obtaining said LASSO-VAR model, where coefficient matrix B(l)∈n×n which represents a coefficient matrix at lag l=1, . . . , p which is split into variables B and H with the added the constraint H=B to obtain model coefficients Bk+1, where
Bk+1=(ZTZ+ρI)−1(ZTY+ρ(Hk−Uk)) (6a)
Hk+1=Sλ/ρ(Bk+1+Uk) (6b)
Uk+1=Uk+Bk+1−Hk+1 (6c)
and where U is the scaled dual variable associated with the constraint H=B, I is the identity matrix with proper dimension, wherein k is an iteration of the optimization method and Sλ/ρ is the soft thresholding operator.

### Claim 11 (depends on 10)

The computer implemented method according to claim 10, further comprising computing BAi in parallel as BAik+1 which is obtained by:
B
A
i
k
+
1
=
arg
⁢
min
B
(

Z
A
i
⁢
B
A
i
k
+
H
_
k
-
ZB
_
k
-
U
k
-
Z
A
i
⁢
B
A
i

2
2
+
λ
⁢

B
A
i

1
)
,
(
8
⁢
a
)
H
_
k
+
1
=
1
N
+
ρ
⁢
(
Y
+
ρ
⁢
ZB
_
k
+
1
+
U
k
)
(
8
⁢
b
)
U
k
+
1
=
U
k
+
ZB
_
k
+
1
-
H
_
k
+
1
,
(
8
⁢
c
)
where
ZB
_
k
+
1
=
1
n
⁢
∑
j
=
1
n
Z
A
j
⁢
B
A
j
k
+
1
,
and BAik+1∈p×n, ZAi∈T×p, T, H, U∈T×n, i=1, . . . , n and BAi is estimated through ADMM by
arg
min
B
(

Y
^
A
i
⁢
–
⁢
Σ
i
⁢
Z
A
i
⁢
B
A
i

2
2
+
λ
^
⁢
Σ
i
⁢

H
A
i

1
)
,
(
9
)
where ŶAi=ZAiBAik+Hk−ZBk−Uk and {circumflex over (λ)}=λ/ρ,
wherein BAi is the non-encrypted solution to the LASSO-VAR model.

### Claim 12 (depends on 11)

The computer implemented method according to claim 11, further comprising obtaining the ADMM solution for the optimization problem:
argmin
B
′
⁢
(
1
2
⁢

MY
-
Σ
i
⁢
MZ
A
i
⁢
Q
A
i
⁢
B
A
i
′

2
2
+
λΣ
i
⁢

Q
A
i
⁢
B
A
i
′

1
)
(
17
)
where B′Ai=QAiBAi is the encrypted solution to the LASSO-VAR model.

### Claim 13 (depends on 1)

The computer implemented method according to claim 1, wherein said linear regression model is for forecasting wind or solar power.

### Claim 14 (depends on 1)

A non-transitory storage media comprising computer program instructions for implementing a method for preserving privacy of a linear regression model used in distributed learning by a set of agents indirectly sharing covariate data or target data when sharing coefficient matrixes for said model, the computer program instructions including instructions which, when executed by a processor of each agent of said set of agents, cause said processors to carry out the method of claim 1.
