# US 2024/0193308 A1 · Methods and systems for federated learning utilizing customer synthetic data models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0193308 A1
- Document ID: US20240193308A1
- Status: Published application
- Kind: A1
- Application number: 18/063,394
- Filed: 2022-12-08
- Published: 2024-06-13
- Earliest claimed: 2022-12-08
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Jeremy GOODSITT; Michael DAVIS; Taylor TURNER; Kenny BEAN; Tyler FARNAN
- Web page: https://patentgrep.com/patent/US20240193308A1

## CPC

- G06F 21/64: Protecting data integrity, e.g. using checksums, certificates or signatures
- G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes}
- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/098: Distributed learning, e.g. federated learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods and systems are described for novel uses and/or improvements to federated learning. As one example, methods and systems are described for improving the applicability of federated learning across various applications and increasing the efficiency of training a global model through federated learning. As another example, methods and systems are described for ensuring comprehensive training data is available to models assigned by the federated learning server. Additionally, methods and systems are described for improving the rate of training a global model through federated learning.

## Claims

### Claim 1

A system for preserving user privacy while generating high-quality training data for federated learning models, the system comprising:
one or more processors; and
a non-transitory computer readable medium having instructions recorded thereon that when executed by the one or more processors cause operations comprising:
retrieving a dataset from a user profile, wherein the user profile is stored locally on a user device;
processing the dataset, by removing anomalies, incomplete data, or outliers, to generate a feature set locally on the user device;
selecting a first feature from the feature set, wherein the first feature is real user data on the local user device to be synthetically generated;
inputting the first feature into a synthetic data generation model, wherein the synthetic data generation model generates a first synthetic output based on real user data on the local user device;
obfuscating the first synthetic output to generate a first synthetic feature, wherein obfuscation techniques involve replacing personally identifiable information with synthetic data;
generating a first synthetic dataset based on the first synthetic feature;
directing a user device to train a machine learning model using the first synthetic dataset; and
transmitting the machine learning model to a centralized remote server for training a federated learning model.

### Claim 2

A method for preserving user privacy while generating high-quality training data for federated learning models, the method comprising:
retrieving a dataset from a user profile, wherein the user profile is stored locally on a user device;
processing the dataset, by removing anomalies, incomplete data, or outliers, to generate a feature set;
selecting a first feature from the feature set;
inputting the first feature into a synthetic data generation model, wherein the synthetic data generation model generates a first synthetic output;
obfuscating the first synthetic output to generate a first synthetic feature;
generating a first synthetic dataset based on the first synthetic feature;
directing a user device to train a machine learning model using the first synthetic dataset; and
transmitting the machine learning model to a centralized remote server from the user device for training a federated learning model.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
generating a second synthetic output based on the first feature; and
obfuscating the second synthetic output to generate a second synthetic feature, wherein the first synthetic dataset is further based on the second synthetic feature.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
inputting a second feature into the synthetic data generation model, wherein the synthetic data generation model generates a second synthetic output; and
obfuscating the second synthetic output to generate a third synthetic feature, wherein the first synthetic dataset is further based on the third synthetic feature.

### Claim 5 (depends on 2)

The method of claim 2, wherein generating the first synthetic output comprises:
determining a plurality of agent outputs based on the first feature; and
aggregating the plurality of agent outputs into the first synthetic output.

### Claim 6 (depends on 2)

The method of claim 2, wherein generating the first synthetic output comprises:
determining a distribution of data based on the first feature; and
determining the first synthetic output based on a likelihood that the distribution of data corresponds to the first synthetic output.

### Claim 7 (depends on 4)

The method of claim 4, wherein generating the first synthetic output comprises:
determining, using a first generative model, a first distribution of data based on the first feature;
determining, using a second generative model, a second distribution of data based on the first feature;
comparing the first distribution to the second distribution;
selecting the first distribution based on comparing the first distribution to the second distribution; and
determining the first synthetic output based on a likelihood that the first distribution of data corresponds to the first synthetic output.

### Claim 8 (depends on 4)

The method of claim 4, wherein generating the first synthetic output comprises:
determining a similar feature based on a manipulation of human language in the first feature; and
determining the first synthetic output based on the similar feature.

### Claim 9 (depends on 2)

The method of claim 2, wherein obfuscating the first synthetic output to generate a first synthetic feature comprises:
determining a secret key based on a random string of bits;
determining an encryption algorithm; and
encrypting the first synthetic output by using the secret key and the encryption algorithm to obfuscate the first synthetic output.

### Claim 10 (depends on 2)

The method of claim 2, wherein obfuscating the first synthetic output to generate a first synthetic feature comprises:
detecting a first text string in the first synthetic output;
determining that the first text string comprises personally identifiable information (PII); and
in response to determining that the first text string comprises PII, replacing the first text string with a second text string.

### Claim 11 (depends on 10)

The method of claim 10, wherein determining that the first text string comprises PII comprises:
comparing the first text string to a list of known instances of PII corresponding to the user device; and
based on comparing the first text string to the list of known instances of PII corresponding to the user device, determining that the first text string corresponds to a first known instance of PII in the list of known instances of PII.

### Claim 12 (depends on 10)

The method of claim 10, wherein determining that the first text string comprises PII comprises:
retrieving a PII text string corresponding to the user device;
comparing the first text string to the PII text string; and
determining that the first text string corresponds to the PII text string.

### Claim 13 (depends on 10)

The method of claim 10, wherein determining that the first text string comprises PII comprises:
determining a first characteristic in the first text string; and
determining a probability that the first text string corresponds to PII based on the first characteristic.

### Claim 14 (depends on 10)

The method of claim 10, wherein replacing the first text string with the second text string further comprising:
determining a first characteristic in the first text string;
determining a data format of the first characteristic;
selecting the second text string based on the data format; and
encrypting the first synthetic output by using a secret key and an encryption algorithm to obfuscate the first synthetic output.

### Claim 15 (depends on 2)

The method of claim 2, wherein obfuscating the first synthetic output to generate a first synthetic feature comprises:
detecting a first text string in the first synthetic output;
detecting a first character in the first text string; and
generating a substitute text string by removing the first character from the first text string.

### Claim 16

A non-transitory, computer-readable medium comprising instructions recorded thereon that when executed by one or more processors cause operations comprising:
retrieving a dataset from a user profile, wherein the user profile is stored locally on a user device;
processing the dataset, by removing anomalies, incomplete data, or outliers, to generate a feature set;
selecting a first feature from the feature set;
inputting the first feature into a synthetic data generation model, wherein the synthetic data generation model generates a first synthetic output;
obfuscating the first synthetic output to generate a first synthetic feature;
generating a first synthetic dataset based on the first synthetic feature;
directing a user device to train a machine learning model using the first synthetic dataset; and
transmitting the model to a centralized remote server from the user device for training a federated learning model.

### Claim 17 (depends on 16)

The non-transitory, computer-readable medium of claim 16, further comprising:
generating a second synthetic output based on the first feature; and
obfuscating the second synthetic output to generate a second synthetic feature, wherein the first synthetic dataset is further based on the second synthetic feature.

### Claim 18 (depends on 16)

The non-transitory, computer-readable medium of claim 16, wherein obfuscating the first synthetic output to generate a first synthetic feature comprises:
determining a secret key based on a random string of bits;
determining an encryption algorithm; and
encrypting the first synthetic output by using the secret key and the encryption algorithm to obfuscate the first synthetic output.

### Claim 19 (depends on 18)

The non-transitory, computer-readable medium of claim 18, wherein obfuscating the first synthetic output to generate a first synthetic feature comprises:
detecting a first text string in the first synthetic output;
determining that the first text string comprises personally identifiable information (PII); and
in response to determining that the first text string comprises PII, replacing the first text string with a second text string.

### Claim 20 (depends on 19)

The non-transitory, computer-readable medium of claim 19, wherein determining that the first text string comprises PII comprises:
comparing the first text string to a list of known instances of PII corresponding to the user device; and
based on comparing the first text string to the list of known instances of PII corresponding to the user device, determining that the first text string corresponds to a first known instance of PII in the list of known instances of PII.
