# US 2024/0005202 A1 · Methods, systems, and media for one-round federated learning with predictive space bayesian inference

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005202 A1
- Document ID: US20240005202A1
- Status: Published application
- Kind: A1
- Application number: 17/963,496
- Filed: 2022-10-11
- Published: 2024-01-04
- Earliest claimed: 2022-05-20
- Inventors: Mohsin HASAN; Zehao ZHANG; Pascal POUPART; Guojun ZHANG; Xi CHEN
- Web page: https://patentgrep.com/patent/US20240005202A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 20/00: Machine learning
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Servers, methods and systems are disclosed for one-round Bayesian federated learning. Embodiments of the present disclosure may assume that each client produces samples from p(y|x, Di) (i.e. the local predictive posteriors), and combines this information to estimate p(y|x, D) (i.e. the global predictive posterior). In some embodiments, an ensemble method may be used that leverages principled Bayesian techniques to incorporate each client's uncertainty estimates.

## Claims

### Claim 1

A method for Bayesian federated learning, comprising:
at each client computing system of a plurality of client computing systems:
obtaining a model space prior comprising a prior probability distribution over a plurality of learnable parameters of a local model of the client computing system;
processing a local dataset of the client computing system to adjust one or more of the plurality of learnable parameters of the local model; and
processing the model space prior and the local model to generate a local predictive posterior; and
aggregating the local predictive posteriors of the plurality of client computing systems to generate a global predictive posterior.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the local predictive posterior is generated using a Markov Chain Monte Carlo algorithm to process the model space prior and the local model.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
obtaining the model space prior comprises receiving the model space prior from a server that computes the model space prior.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the model space prior is a predetermined model space prior; and
obtaining the model space prior comprises retrieving the predetermined model space prior from a memory of the client computing system.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
aggregating the local predictive posteriors comprises:
sending the local predictive posteriors of the plurality of client computing systems to a server to generate the global predictive posterior.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
aggregating the local predictive posteriors comprises:
receiving, at a first client computing system of the plurality of client computing systems, the local predictive posteriors of the plurality of client computing systems; and
processing, at the first client computing system, the plurality of local predictive posteriors to generate the global predictive posterior.

### Claim 7 (depends on 1)

The method of claim 1, wherein:
the local predictive posterior comprises a plurality of posterior probability samples over a corresponding plurality of query inputs.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
the plurality of query inputs used by each client computing system are obtained from a shared data set; and
each client computing system obtains the shared data set from a server.

### Claim 9 (depends on 1)

The method of claim 1, wherein:
aggregating the local predictive posteriors comprises using Gaussian approximation to:
for each client computing system, process the respective local predictive posterior to estimate a respective sample mean and covariance; and
process the sample means and covariances for the plurality of client computing systems to estimate a mean and covariance of the global predictive posterior.

### Claim 10 (depends on 9)

The method of claim 9, wherein:
the global predictive posterior comprises a regression prediction; and
processing the sample means and covariances comprises:
averaging the sample means using a weight based on the covariances.

### Claim 11 (depends on 1)

The method of claim 1, wherein:
aggregating the local predictive posteriors comprises using a Kernel Density Estimator to:
for each client computing system, process a plurality of samples of the respective local predictive posterior to estimate a density of the respective local predictive posterior; and
process the estimated densities for the plurality of client computing systems, using an optimization algorithm, to estimate the global predictive posterior.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
generating a trained global model based on the global predictive posterior.

### Claim 13 (depends on 12)

The method of claim 12, wherein:
generating the trained global model comprises training the global model to approximate the global predictive posterior, on a server, using knowledge distillation; and
the method further comprises communicating the trained global model to each client computing system of the plurality of client computing systems.

### Claim 14

A computing system comprising:
a processing device; and
a memory storing thereon:
a local model comprising a plurality of learnable parameters;
a local dataset; and
machine-executable instructions which, when executed by the processing device, cause the computing system to perform Bayesian federated learning by:
obtaining a model space prior comprising a prior probability distribution over the plurality of learnable parameters;
processing the local dataset to adjust one or more of the plurality of learnable parameters;
processing the model space prior and the local model to generate a local predictive posterior;
obtaining a local predictive posterior of each client computing system of a plurality of client computing systems; and
aggregating the local predictive posteriors of the computing system and the plurality of client computing systems to generate a global predictive posterior.

### Claim 15 (depends on 14)

The computing system of claim 14, wherein:
aggregating the local predictive posteriors comprises using Gaussian approximation to:
for the computing system and each client computing system, process the respective local predictive posterior to estimate a respective sample mean and covariance; and
process the sample means and covariances for the plurality of client computing systems to estimate a mode of the global predictive posterior.

### Claim 16 (depends on 14)

The computing system of claim 14, wherein:
aggregating the local predictive posteriors comprises using a Kernel Density Estimator to:
for each client computing system, process a plurality of samples of the respective local predictive posterior to estimate a density of the respective local predictive posterior; and
process the estimated densities for the plurality of client computing systems, using an optimization algorithm, to estimate the global predictive posterior.

### Claim 17

A server comprising:
a processing device; and
a memory storing thereon machine-executable instructions which, when executed by the processing device, cause the server to perform Bayesian federated learning by:
obtaining a local predictive posterior of each client computing system of a plurality of client computing systems; and
aggregating the local predictive posteriors of the computing system and the plurality of client computing systems to generate a global predictive posterior.

### Claim 18 (depends on 17)

The server of claim 17, wherein:
aggregating the local predictive posteriors comprises using Gaussian approximation to:
for each client computing system, process the respective local predictive posterior to estimate a respective sample mean and covariance; and
process the sample means and covariances for the plurality of client computing systems to estimate a mean and covariance of the global predictive posterior.

### Claim 19 (depends on 17)

The server of claim 17, wherein:
aggregating the local predictive posteriors comprises using a Kernel Density Estimator to:
for each client computing system, process a plurality of samples of the respective local predictive posterior to estimate a density of the respective local predictive posterior; and
process the estimated densities for the plurality of client computing systems, using an optimization algorithm, to estimate the global predictive posterior.

### Claim 20 (depends on 1)

A non-transitory processor-readable medium having machine-executable instructions stored thereon which, when executed by a processing device of a computing system, cause the computing system to perform the steps of the method of claim 1.
