# US 2025/0158638 A1 · Federated latent transformer deep learning core

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0158638 A1
- Document ID: US20250158638A1
- Status: Published application
- Kind: A1
- Application number: 19/026,345
- Filed: 2025-01-16
- Published: 2025-05-15
- Earliest claimed: 2017-10-30
- Assignees: Atombeam Technologies Inc.
- Applicants: AtomBeam Technologies Inc.
- Inventors: Brian Galvin
- Web page: https://patentgrep.com/patent/US20250158638A1

## CPC

- H03M 7/3059: {Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression}
- G06N 3/045: Combinations of networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 20/00: Machine learning
- H03M 7/6005: {Decoder aspects}
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system and method for a federated deep learning platform utilizing homomorphically-compressed and encrypted data. The system comprises multiple client devices, each with a local dataset, and a central server hosting a deep learning core. Client devices convert local data into codewords, which are also homomorphically encrypted. The central server processes these encrypted codewords without decryption, preserving data privacy. The platform supports at least two architectural variants: a conventional Transformer trained on codewords, and a Latent Transformer operating on latent space vectors. Both variants eliminate the need for embedding and positional encoding layers. The system aggregates encrypted model updates from clients, enabling collaborative learning while maintaining data confidentiality. Additional features comprise differential privacy implementation and adaptive federated optimization techniques. This innovative approach allows for efficient, privacy-preserving distributed learning across diverse datasets, addressing key challenges in federated learning such as data heterogeneity, non-IID distributions, and communication efficiency.

## Claims

### Claim 1

A system for federated deep learning using a latent transformer core, comprising:
a computing device comprising at least a memory and a processor;
a plurality of programming instructions that, when operating on the processor, cause the computing device to:
receive as input a plurality of bytes from a plurality of client devices;
encode the sequence of bytes into latent space vectors using an encoder;
process the latent space vectors using a deep learning core; and
decode the processed latent space vectors into output bytes using a decoder;
wherein the trained machine learning core was initially trained, using training inputs comprising encoded latent space vectors, to predict a plurality of probable future byte sequences.

### Claim 2 (depends on 1)

The system of claim 1, wherein the deep learning core comprises a latent transformer architecture.

### Claim 3 (depends on 2)

The system of claim 2, wherein each client device further comprises a variational autoencoder encoder that generates latent space vectors from the plurality of bytes.

### Claim 4 (depends on 3)

The system of claim 3, wherein the latent transformer architecture processes the latent space vectors.

### Claim 5 (depends on 1)

The system of claim 1, wherein the plurality of programming instructions further causes the computing device to:
aggregate encrypted model updates from the plurality of client devices;
update the deep learning core based on the aggregated encrypted model updates; and
facilitate federated learning by iteratively updating the deep learning core based on encrypted updates from the client devices.

### Claim 6 (depends on 5)

The system of claim 5, wherein the plurality of programming instructions further causes the computing device to:
implement differential privacy by:
adding calibrated noise to the encrypted model updates before aggregation;
enforcing a privacy budget across multiple rounds of federated learning; and
dynamically adjusting the level of noise based on the privacy budget consumption;
thereby enhancing privacy guarantees for individual client datasets while maintaining model utility.

### Claim 7

A method for federated deep learning using homomorphically-compressed and encrypted data, comprising the steps of:
receiving as input a plurality of bytes from a plurality of client devices;
encoding the sequence of bytes into latent space vectors using an encoder;
processing the latent space vectors using a deep learning core; and
decoding the processed latent space vectors into output bytes using a decoder;
wherein the trained machine learning core was initially trained, using training inputs comprising encoded latent space vectors, to predict a plurality of probable future byte sequences.

### Claim 8 (depends on 7)

The method of claim 7, wherein the deep learning core comprises a latent transformer architecture.

### Claim 9 (depends on 8)

The method of claim 8, wherein each client device further comprises a variational autoencoder encoder that generates latent space vectors from the plurality of bytes.

### Claim 10 (depends on 9)

The method of claim 9, wherein the latent transformer architecture processes the latent space vectors.

### Claim 11 (depends on 10)

The method of claim 10, further comprising the step of generating output vectors from processed latent space vectors using a variational autoencoder decoder.

### Claim 12 (depends on 7)

The method of claim 7, further comprising the steps of:
aggregating encrypted model updates from the plurality of client devices;
updating the deep learning core based on the aggregated encrypted model updates; and
facilitating federated learning by iteratively updating the deep learning core based on encrypted updates from the client devices.

### Claim 13 (depends on 12)

The method of claim 12, further comprising the steps of:
implementing differential privacy by:
adding calibrated noise to the encrypted model updates before aggregation;
enforcing a privacy budget across multiple rounds of federated learning; and
dynamically adjusting the level of noise based on the privacy budget consumption;
thereby enhancing privacy guarantees for individual client datasets while maintaining model utility.
