# US 12,182,272 B2 · Method and apparatus for actively defending against backdoor attacks under edge computing scenario

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,272 B2
- Document ID: US12182272B2
- Status: Granted
- Kind: B2
- Application number: 17/523,474
- Filed: 2021-11-10
- Published: 2022-07-14
- Granted: 2024-12-31
- Earliest claimed: 2021-01-13
- Assignees: Tsinghua University
- Applicants: Tsinghua University
- Inventors: Ke Xu; Yi Zhao; Su Yao; Ziwei Li
- Web page: https://patentgrep.com/patent/US12182272B2

## CPC

- G06F 21/577: {Assessing vulnerabilities and evaluating computer system security}
- G06F 21/54: by adding security routines or objects to programs
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 3/12: using genetic models
- G06N 3/126: Evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method and an apparatus for actively defending against backdoor attacks under an edge computing scenario. The method includes: generating an initial optional operation set based on optional operations configured to enhance a model generalization ability, and constructing a configuration parameter set for each operation in the set; selecting a first subset with a curve between configuration parameters and model accuracies of the operation presenting a monotonically decreasing concave function characteristic; selecting a second subset with a curve of the operation between configuration parameters and probabilities that a model is successfully attacked by the backdoor attack presenting a monotonically decreasing convex function characteristic; constructing a mutation characteristic set for each operation in an intersection of the first and second subsets; and for each operation in the intersection, determining a final parameter setting value of the operation based on the mutation characteristic set, to actively defend against possible backdoor attacks.

## Claims

### Claim 1

A method for actively defending against backdoor attacks under an edge computing scenario, performed by a collaborative learning edge node under the edge computing scenario, the edge node being deployed a deep learning model, the method comprising:
generating an initial optional operation set based on optional operations configured to enhance a model generalization ability, and constructing a configuration parameter set for each optional operation in the initial optional operation set;
selecting a first optional operation subset from the initial optional operation set, wherein for each optional operation in the first optional operation subset, a curve between configuration parameters and model accuracies presents a monotonically decreasing concave function characteristic;
selecting a second optional operation subset from the initial optional operation set, wherein for each optional operation in the second optional operation subset, a curve between configuration parameters and probabilities that the deep learning model is successfully attacked by the backdoor attack presents a monotonically decreasing convex function characteristic;
constructing a mutation characteristic set for each optional operation in an intersection of the first optional operation subset and the second optional operation subset according to a preset formula;
for each optional operation in the intersection of the first optional operation subset and the second optional operation subset, determining a final parameter setting value of the optional operation based on the mutation characteristic set; and
actively defending against possible backdoor attacks by training and using the deep learning model with a parameter of the optional operation being set to the final parameter setting value, wherein the final parameter setting value decreases a probability of the deep learning model being embedded with an effective backdoor.

### Claim 2 (depends on 1)

The method of claim 1, wherein, constructing the configuration parameter set for each optional operation in the initial optional operation set, comprises:
for each optional operation in the initial optional operation set, selecting K+2 parameters evenly from an optional parameter configuration range L={lmini, lmaxi} of the optional operation, where lmini<lmaxi, lmini is the minimum optional configuration parameter and lmaxi is the maximum optional configuration parameter, and the K+2 parameters include lmini and lmaxi, and K is an integer greater than or equal to 0.

### Claim 3 (depends on 1)

The method of claim 1, wherein, selecting the first optional operation subset from the initial optional operation set comprises:
for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and model accuracies;
for each optional operation in the initial optional operation set, constructing the curve between configuration parameters and model accuracies using the two-tuple set, and generating an approximate curvilinear function by fitting; and
selecting a subset of optional operations corresponding to the curvilinear functions conforming to a preset monotonically decreasing concave function characteristic, as the first optional operation subset.

### Claim 4 (depends on 3)

The method of claim 3, wherein for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and model accuracies, comprises:
for each optional operation in the initial optional operation set, training the model using each configuration parameter in the configuration parameter set corresponding to the optional operation;
testing the model after training the model to obtain the model accuracy;
constructing a two-tuple between the configuration parameter and the model accuracy; and
forming the two-tuple set with respective two-tuples corresponding to respective configuration parameters.

### Claim 5 (depends on 1)

The method of claim 1, wherein, selecting the second optional operation subset from the initial optional operation set comprises:
for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and probabilities that the model is successfully attacked;
for each optional operation in the initial optional operation set, constructing the curve between configuration parameters and probabilities that the model is successfully attacked using the two-tuple set, and generating an approximate curvilinear function by fitting; and
selecting a subset of optional operations corresponding to the curvilinear functions conforming to a preset monotonically decreasing convex function characteristic, as the second optional operation subset.

### Claim 6 (depends on 5)

The method of claim 5, wherein for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and probabilities that the model is successfully attacked, comprises:
for each optional operation in the initial optional operation set, training the model using each configuration parameter in the configuration parameter set corresponding to the optional operation;
testing the model using samples with a backdoor attack trigger after training the model, to obtain the probability that the model is successfully attacked;
constructing a two-tuple between the configuration parameter and the probability that the model is successfully attacked; and
forming the two-tuple set with respective two-tuples corresponding to respective configuration parameters.

### Claim 7 (depends on 1)

The method of claim 1, wherein, constructing the mutation characteristic set for each optional operation in the intersection of the first optional operation subset and the second optional operation subset according to the preset formula, comprises:
constructing a finally selected optional operation set, wherein the finally selected optional operation set is the intersection of the first optional operation subset and the second optional operation subset;
for each operation in the finally selected optional operation set, constructing a two-tuple set of configuration parameters and model generalization error upper bounds; and
constructing the mutation characteristic set for each operation in the finally selected optional operation set based on the two-tuple set.

### Claim 8 (depends on 7)

The method of claim 7, wherein constructing the mutation characteristic set for each operation in the finally selected optional operation set based on the two-tuple set comprises:
for each two-tuple in the two-tuple set, defining the mutation characteristic according to a formula of
w
j
i
=
{

ϕ
j
i
-
ϕ
j
+
Δ
i

e
j
i
-
e
j
+
Δ
i
,
j
+
Δ
≤
K
+
2
0
,
otherwise
where,
Vi={v1i=(e1i, ϕ1i), v2i=(e2i, ϕ2i), . . . , vji=(eji, ϕji), vj+1i=(ej+1i, ϕj+1i), . . . , vK+1i=(eK+1i, ϕK+1i), vK+2i=(eK+2i, ϕK+2i)} is the set of all two-tuples, ϕji is the model generalization error upper bound, eji is the configuration parameter in the configuration parameter set corresponding to the operation, Δ is an integer hyper-parameter greater than or equal to 1, and K is an integer greater than or equal to 0; and
constructing the mutation characteristic set as Wi={w1i, w2i, . . . , wji, wj+1i, . . . , wK+1i, wK+2i}.

### Claim 9 (depends on 8)

The method of claim 8, wherein, for each optional operation in the intersection of the first optional operation subset and the second optional operation subset, determining a final parameter setting value of the optional operation based on the corresponding mutation characteristic set, comprises:
for the mutation characteristic set associated with each optional operation in the intersection of the first optional operation subset and the second optional operation subset, comparing wji with a preset threshold starting from j=1 and gradually increasing by 1; and
determining the parameter value corresponding to wji in response to wji being greater than the preset threshold, as the final parameter setting value.

### Claim 10

An apparatus for actively defending against backdoor attacks under an edge computing scenario, applied to a collaborative learning edge node under the edge computing scenario, the edge node being deployed a deep learning model, the apparatus comprising:
a processor; and
a memory, configured to store instructions executable by the processor,
wherein the processor is configured to implement a method for actively defending against backdoor attacks under an edge computing scenario by executing the instructions stored in the memory, the method comprising:
generating an initial optional operation set based on optional operations configured to enhance a model generalization ability, and constructing a configuration parameter set for each optional operation in the initial optional operation set;
selecting a first optional operation subset from the initial optional operation set, wherein for each optional operation in the first optional operation subset, a curve between configuration parameters and model accuracies presents a monotonically decreasing concave function characteristic;
selecting a second optional operation subset from the initial optional operation set, wherein for each optional operation in the second optional operation subset, a curve between configuration parameters and probabilities that the deep learning model is successfully attacked by a backdoor attack presents a monotonically decreasing convex function characteristic;
constructing a mutation characteristic set for each optional operation in an intersection of the first optional operation subset and the second optional operation subset according to a preset formula;
for each optional operation in the intersection of the first optional operation subset and the second optional operation subset, determining a final parameter setting value of the optional operation based on the mutation characteristic set; and
actively defending against possible backdoor attacks by training and using the deep learning model with a parameter of the optional operation being set to the final parameter setting value, wherein the final parameter setting value decreases a probability of the deep learning model being embedded with an effective backdoor.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein, constructing the configuration parameter set for each optional operation in the initial optional operation set, comprises:
for each optional operation in the initial optional operation set, selecting K+2 parameters evenly from an optional parameter configuration range L={lmini, lmaxi} of the optional operation, where lmini<lmaxi, lmini is the minimum optional configuration parameter and lmaxi is the maximum optional configuration parameter, and the K+2 parameters include lmini and lmaxi, and K is an integer greater than or equal to 0.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein, selecting the first optional operation subset from the initial optional operation set comprises:
for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and model accuracies;
for each optional operation in the initial optional operation set, constructing the curve between configuration parameters and model accuracies using the two-tuple set, and generating an approximate curvilinear function by fitting; and
selecting a subset of optional operations corresponding to the curvilinear functions conforming to a preset monotonically decreasing concave function characteristic, as the first optional operation subset.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and model accuracies, comprises:
for each optional operation in the initial optional operation set, training the model using each configuration parameter in the configuration parameter set corresponding to the optional operation;
testing the model after training the model to obtain the model accuracy;
constructing a two-tuple between the configuration parameter and the model accuracy; and
forming the two-tuple set with respective two-tuples corresponding to respective configuration parameters.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein, selecting the second optional operation subset from the initial optional operation set comprises:
for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and probabilities that the model is successfully attacked;
for each optional operation in the initial optional operation set, constructing the curve between configuration parameters and probabilities that the model is successfully attacked using the two-tuple set, and generating an approximate curvilinear function by fitting; and
selecting a subset of optional operations corresponding to the curvilinear functions conforming to a preset monotonically decreasing convex function characteristic, as the second optional operation subset.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein for each optional operation in the initial optional operation set, constructing a two-tuple set of configuration parameters and probabilities that the model is successfully attacked, comprises:
for each optional operation in the initial optional operation set, training the model using each configuration parameter in the configuration parameter set corresponding to the optional operation;
testing the model using samples with a backdoor attack trigger after training the model, to obtain the probability that the model is successfully attacked;
constructing a two-tuple between the configuration parameter and the probability that the model is successfully attacked; and
forming the two-tuple set with respective two-tuples corresponding to respective configuration parameters.

### Claim 16 (depends on 10)

The apparatus of claim 10, wherein, constructing the mutation characteristic set for each optional operation in the intersection of the first optional operation subset and the second optional operation subset according to the preset formula, comprises:
constructing a finally selected optional operation set, wherein the finally selected optional operation set is the intersection of the first optional operation subset and the second optional operation subset;
for each operation in the finally selected optional operation set, constructing a two-tuple set of configuration parameters and model generalization error upper bounds; and
constructing the mutation characteristic set for each operation in the finally selected optional operation set based on the two-tuple set.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein constructing the mutation characteristic set for each operation in the finally selected optional operation set based on the two-tuple set comprises:
for each two-tuple in the two-tuple set, defining the mutation characteristic according to a formula of
w
j
i
=
{

ϕ
j
i
-
ϕ
j
+
Δ
i

e
j
i
-
e
j
+
Δ
i
,
j
+
Δ
≤
K
+
2
0
,
otherwise
where,
Vi={v1i=(e1i, ϕ1i), v2i=(e2i, ϕ2i), . . . , vji=(eji, ϕji), vj+1i=(ej+1i, ϕj+1i), . . . , vK+1i=(eK+1i, ϕK+1i), vK+2i=(eK+2i, ϕK+2i)} is the set of all two-tuples, ϕji is the model generalization error upper bound, eji is the configuration parameter in the configuration parameter set corresponding to the operation, Δ is an integer hyper-parameter greater than or equal to 1, and K is an integer greater than or equal to 0; and
constructing the mutation characteristic set as Wi={w1i, w2i, . . . , wji, wj+1i, . . . , wK+1i, wK+2i}.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein, for each optional operation in the intersection of the first optional operation subset and the second optional operation subset, determining a final parameter setting value of the optional operation based on the corresponding mutation characteristic set, comprises:
for the mutation characteristic set associated with each optional operation in the intersection of the first optional operation subset and the second optional operation subset, comparing wji with a preset threshold starting from j=1 and gradually increasing by 1; and
determining the parameter value corresponding to wji in response to wji being greater than the preset threshold, as the final parameter setting value.

### Claim 19

A non-transitory computer readable storage medium stored thereon with a computer program that, when executed by a processor applied to a collaborative learning edge node under the edge computing scenario, cause the processor to implement a method for actively defending against backdoor attacks under an edge computing scenario, the edge node being deployed a deep learning model, the method comprising:
generating an initial optional operation set based on optional operations configured to enhance a model generalization ability, and constructing a configuration parameter set for each optional operation in the initial optional operation set;
selecting a first optional operation subset from the initial optional operation set, wherein for each optional operation in the first optional operation subset, a curve between configuration parameters and model accuracies presents a monotonically decreasing concave function characteristic;
selecting a second optional operation subset from the initial optional operation set, wherein for each optional operation in the second optional operation subset, a curve between configuration parameters and probabilities that the deep learning model is successfully attacked by the backdoor attack presents a monotonically decreasing convex function characteristic;
constructing a mutation characteristic set for each optional operation in an intersection of the first optional operation subset and the second optional operation subset according to a preset formula;
for each optional operation in the intersection of the first optional operation subset and the second optional operation subset, determining a final parameter setting value of the optional operation based on the mutation characteristic set; and
actively defend against possible backdoor attacks by training and using the deep learning model with a parameter of the optional operation being set to the final parameter setting value, wherein the final parameter setting value decreases a probability of the deep learning model being embedded with an effective backdoor.