# US 2026/0030552 A1 · Privacy erase model training method and apparatus and privacy erase method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0030552 A1
- Document ID: US20260030552A1
- Status: Published application
- Kind: A1
- Application number: 19/283,017
- Filed: 2025-07-28
- Published: 2026-01-29
- Earliest claimed: 2024-07-29
- Applicants: Ant Group Co., Ltd.
- Inventors: Yan Liu; Haiqin Weng
- Web page: https://patentgrep.com/patent/US20260030552A1

## CPC

- G06N 20/00: Machine learning
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes}
- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In a privacy erase model training solution, a large language model is separately trained based on original training data and anonymized data of the original training data by using completely same training methods, and weights of the same large language model on the original training data and the anonymized data are recorded, to form a new erase data training set. A privacy erase model can be trained by using the erase data training set, to erase weight data related to privacy data from the large language model. The privacy erase model is trained to directly modify a parameter of the large language model, so that the privacy data in memory of the large language model is fundamentally deleted, thereby achieving extremely high security.

## Claims

### Claim 1

A method, comprising:
obtaining at least one large language model to be trained;
obtaining a first training sample set that includes privacy data;
performing anonymization processing on the privacy data in the first training sample set to obtain an anonymized second training sample set;
for each large language model of the at least one large language model, training the large language model by using the first training sample set, and obtaining first weight data of the large language model after the large language model is trained by using the first training sample set;
training the large language model by using the second training sample set and by using a training process consistent with that of the training the large language model by using the first training sample set, and obtaining second weight data of the large language model after the large language model is trained by using the second training sample set; and
training a privacy erase model by using a labelled training set including a model structure description text of the large language model, the first weight data of the large language model, and the second weight data of the large language model.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the training the privacy erase model includes:
inputting the model structure description text of the large language model and the first weight data of the large language model to the privacy erase model to obtain model weight data predicted by the privacy erase model;
constructing a loss function based on the model weight data predicted by the privacy erase model and the second weight data of the large language model; and
updating a parameter of the privacy erase model based on the loss function.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the loss function is constructed based on a cross entropy between the model weight data predicted by the privacy erase model and the second weight data of the large language model.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the training the privacy erase model includes using the model structure description text and the first weight data as input samples, and using the second weight data as output labels.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising:
obtaining initial weight data of a target large language model after the target large language model is trained based on a training task;
inputting a target model structure description text of the target large language model and the initial weight data to the privacy erase model, to obtain privacy-erased weight data; and
reconstructing the target large language model based on the target model structure description text and the privacy-erased weight data.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the obtaining the initial weight data after the target large language model is trained based on the training task includes:
obtaining a target training sample set based on the training task of the target large language model;
training the target large language model by using the target training sample set; and
obtaining the initial weight data of the trained target large language model after the training the target large language model by using the target training sample set.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the first weight data includes a first weight coefficient of the large language model and the second weight data includes a second weight coefficient of the large language model.

### Claim 8

A computing system including one or more processors and one or more storage devices, the one or more storage device, individually or collectively, having executable instructions stored thereon, the executable instructions, when executed by the one or more processors, enabling the one or more processors to, individually or collectively, implement acts comprising:
obtaining at least one large language model to be trained;
obtaining a first training sample set that includes privacy data;
performing anonymization processing on the privacy data in the first training sample set to obtain an anonymized second training sample set;
for each large language model of the at least one large language model, training the large language model by using the first training sample set, and obtaining first weight data of the large language model after the large language model is trained by using the first training sample set;
training the large language model by using the second training sample set and by using a training process consistent with that of the training the large language model by using the first training sample set, and obtaining second weight data of the large language model after the large language model is trained by using the second training sample set; and
training a privacy erase model by using a labelled training set including a model structure description text of the large language model, the first weight data of the large language model, and the second weight data of the large language model.

### Claim 9 (depends on 8)

The computing system according to claim 8, wherein the training the privacy erase model includes:
inputting the model structure description text of the large language model and the first weight data of the large language model to the privacy erase model to obtain model weight data predicted by the privacy erase model;
constructing a loss function based on the model weight data predicted by the privacy erase model and the second weight data of the large language model; and
updating a parameter of the privacy erase model based on the loss function.

### Claim 10 (depends on 9)

The computing system according to claim 9, wherein the loss function is constructed based on a cross entropy between the model weight data predicted by the privacy erase model and the second weight data of the large language model.

### Claim 11 (depends on 8)

The computing system according to claim 8, wherein the training the privacy erase model includes using the model structure description text and the first weight data as input samples, and using the second weight data as output labels.

### Claim 12 (depends on 8)

The computing system according to claim 8, wherein the acts further include:
obtaining initial weight data of a target large language model after the target large language model is trained based on a training task;
inputting a target model structure description text of the target large language model and the initial weight data to the privacy erase model, to obtain privacy-erased weight data; and
reconstructing the target large language model based on the target model structure description text and the privacy-erased weight data.

### Claim 13 (depends on 12)

The computing system according to claim 12, wherein the obtaining the initial weight data after the target large language model is trained based on the training task includes:
obtaining a target training sample set based on the training task of the target large language model;
training the target large language model by using the target training sample set; and
obtaining the initial weight data of the trained target large language model after the training the target large language model by using the target training sample set.

### Claim 14 (depends on 8)

The computing system according to claim 8, wherein the first weight data includes a first weight coefficient of the large language model and the second weight data includes a second weight coefficient of the large language model.

### Claim 15

A non-transitory computer readable medium having executable instructions stored thereon, the executable instructions, when executed by one or more processors, enabling the one or more processors to, individually or collectively, implement acts comprising:
obtaining at least one large language model to be trained;
obtaining a first training sample set that includes privacy data;
performing anonymization processing on the privacy data in the first training sample set to obtain an anonymized second training sample set;
for each large language model of the at least one large language model, training the large language model by using the first training sample set, and obtaining first weight data of the large language model after the large language model is trained by using the first training sample set;
training the large language model by using the second training sample set and by using a training process consistent with that of the training the large language model by using the first training sample set, and obtaining second weight data of the large language model after the large language model is trained by using the second training sample set; and
training a privacy erase model by using a labelled training set including a model structure description text of the large language model, the first weight data of the large language model, and the second weight data of the large language model.

### Claim 16 (depends on 15)

The non-transitory computer readable medium according to claim 15, wherein the training the privacy erase model includes:
inputting the model structure description text of the large language model and the first weight data of the large language model to the privacy erase model to obtain model weight data predicted by the privacy erase model;
constructing a loss function based on the model weight data predicted by the privacy erase model and the second weight data of the large language model; and
updating a parameter of the privacy erase model based on the loss function.

### Claim 17 (depends on 16)

The non-transitory computer readable medium according to claim 16, wherein the loss function is constructed based on a cross entropy between the model weight data predicted by the privacy erase model and the second weight data of the large language model.

### Claim 18 (depends on 15)

The non-transitory computer readable medium according to claim 15, wherein the training the privacy erase model includes using the model structure description text and the first weight data as input samples, and using the second weight data as output labels.

### Claim 19 (depends on 15)

The non-transitory computer readable medium according to claim 15, wherein the acts further include:
obtaining initial weight data of a target large language model after the target large language model is trained based on a training task;
inputting a target model structure description text of the target large language model and the initial weight data to the privacy erase model, to obtain privacy-erased weight data; and
reconstructing the target large language model based on the target model structure description text and the privacy-erased weight data.

### Claim 20 (depends on 19)

The non-transitory computer readable medium according to claim 19, wherein the obtaining the initial weight data after the target large language model is trained based on the training task includes:
obtaining a target training sample set based on the training task of the target large language model;
training the target large language model by using the target training sample set; and
obtaining the initial weight data of the trained target large language model after the training the target large language model by using the target training sample set.
