# US 11,443,178 B2 · Deep neural network hardening framework

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,178 B2
- Document ID: US11443178B2
- Status: Granted
- Kind: B2
- Application number: 15/844,442
- Filed: 2017-12-15
- Published: 2019-06-20
- Granted: 2022-09-13
- Earliest claimed: 2017-12-15
- Assignees: Interntional Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Benjamin J. Edwards; Taesung Lee; Ian M. Molloy; Dong Su
- Web page: https://patentgrep.com/patent/US11443178B2

## CPC

- G06N 3/094: Adversarial learning
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- H04L 63/1441: {Countermeasures against malicious traffic (countermeasures against attacks on cryptographic mechanisms H04L9/002)}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Mechanisms are provided to implement a hardened neural network framework. A data processing system is configured to implement a hardened neural network engine that operates on a neural network to harden the neural network against evasion attacks and generates a hardened neural network. The hardened neural network engine generates a reference training data set based on an original training data set. The neural network processes the original training data set and the reference training data set to generate first and second output data sets. The hardened neural network engine calculates a modified loss function of the neural network, where the modified loss function is a combination of an original loss function associated with the neural network and a function of the first and second output data sets. The hardened neural network engine trains the neural network based on the modified loss function to generate the hardened neural network.

## Claims

### Claim 1

A method, in a data processing system comprising a processor and a memory, the memory comprising instructions which are executed by the processor to specifically configure the processor to implement a hardened neural network, the method comprising:
configuring the data processing system to implement a hardened neural network engine that operates on a neural network to harden the neural network against evasion attacks and generates a hardened neural network;
generating, by the hardened neural network engine, a reference training data set based on an original training data set;
processing, by the neural network, the original training data set to generate a first output data set and processing the reference training data set to generate a second output data set;
calculating, by the hardened neural network engine, a modified loss function of the neural network, wherein the modified loss function is a combination of an original loss function associated with the neural network, and a delta function of a ratio of a similarity between the original training data set and the reference training data set and a similarity between the first output data set and the second output data set; and
training, by the hardened neural network engine, the neural network based on the modified loss function to generate the hardened neural network, wherein the delta function, Delta, is as follows:
Delta(X,X′,Y,Y′)=cos(X,X′)/cos(Y,Y′),
wherein X is the original training data set, X′ is the reference training data set, Y is the first output data set, Y′ is the second output data set, and cos is a cosine similarity function.

### Claim 2 (depends on 1)

The method of claim 1, wherein the modified loss function causes differences between data in the original training data set and data in the reference training data set to be commensurate with differences between the corresponding outputs of the hardened neural network.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the reference training data set comprises at least one of performing a perturbation operation on the original training data set to introduce perturbations in the original training data set and generate the reference training data set comprising the original training data set with the introduced perturbations, or performing a sampling operation to sample data from the original training data set to generate the reference training data set.

### Claim 4 (depends on 1)

The method of claim 1, wherein training the neural network based on the modified loss function to generate the hardened neural network comprises training the neural network to minimize both the delta function and the original loss function associated with the neural network.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
performing, by the hardened neural network engine, a classification operation to classify an input data into one of a plurality of predefined classes of input data; and
performing, by a cognitive computing system, a reasoning operation based on results of the classification operation.

### Claim 6 (depends on 5)

The method of claim 5, wherein performing the reasoning operation based on the results of the classification operation comprises annotating the input data to include a class corresponding to a predefined class having a highest probability value associated with the class as determined by the hardened neural network engine.

### Claim 7 (depends on 1)

The method of claim 1, wherein the hardened neural network is hardened against gradient based attacks such that perturbations in input data to the hardened neural network do not cause misclassification by the hardened neural network.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
receiving, by the hardened neural network, input data to be processed;
generating, by the hardened neural network, an output vector comprising a plurality of probability values stored in vector slots, wherein each vector slot is associated with a different class in a plurality of predefined classes such that the probability value stored in a vector slot indicates a probability that the input data is properly classified into a class corresponding to the vector slot;
providing, by the hardened neural network, the output vector to a cognitive system; and
executing, by the cognitive system, a cognitive operation based on the probability values stored in the output vector.

### Claim 9 (depends on 8)

The method of claim 8, wherein the input data is a data sample of at least one of image data, audio data, or textual data, and wherein the cognitive model logic operates on the data sample to classify the data sample according to the cognitive operation of the cognitive system, and wherein the cognitive operation is one of an image analysis operation, a speech recognition operation, an audio recognition operation, a social network filtering operation, a machine translation operation, a natural language processing operation, a patient treatment recommendation operation, a medical imaging analysis operation, or a bioinformatics operation.

### Claim 10 (depends on 1)

The method of claim 1, wherein the modified loss function, L′, is as follows:
L′=L+Delta(X,X′,Y,Y′),
wherein L is the original loss function associated with the neural network.

### Claim 11

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
configure the data processing system to implement a hardened neural network engine that operates on a neural network to harden the neural network against evasion attacks and generates a hardened neural network;
generate a reference training data set based on an original training data set;
process, by the neural network, the original training data set and the reference training data set to generate first and second output data sets;
calculate, by the hardened neural network engine, a modified loss function of the neural network, wherein the modified loss function is a combination of an original loss function associated with the neural network, and a delta function of a ratio of a similarity between the original training data set and the reference data set and a similarity between the first output data set and the second output data set; and
train, by the hardened neural network engine, the neural network based on the modified loss function to generate the hardened neural network, wherein the delta function, Delta, is as follows:
Delta(X,X′,Y,Y′)=cos(X,X′)/cos(Y,Y′),
wherein X is the original training data set, X′ is the reference training data set, Y is the first output data set, Y′ is the second output data set, and cos is a cosine similarity function.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein the modified loss function causes differences between data in the original training data set and data in the reference training data set to be commensurate with differences between the corresponding outputs of the hardened neural network.

### Claim 13 (depends on 11)

The computer program product of claim 11, wherein the computer readable program further causes the data processing system to generate the reference training data set at least by one of performing a perturbation operation on the original training data set to introduce perturbations in the original training data set and generate the reference training data set comprising the original training data set with the introduced perturbations, or performing a sampling operation to sample data from the original training data set to generate the reference training data set.

### Claim 14 (depends on 11)

The computer program product of claim 11, wherein the computer readable program further causes the data processing system to train the neural network based on the modified loss function to generate the hardened neural network at least by training the neural network to minimize both the delta function and the original loss function associated with the neural network.

### Claim 15 (depends on 11)

The computer program product of claim 11, wherein the computer readable program further causes the data processing system to:
perform, by the hardened neural network engine, a classification operation to classify an input data into one of a plurality of predefined classes of input data; and
perform, by a cognitive computing system, a reasoning operation based on results of the classification operation.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the computer readable program further causes the data processing system to perform the reasoning operation based on the results of the classification operation at least by annotating the input data to include a class corresponding to a predefined class having a highest probability value associated with the class as determined by the hardened neural network engine.

### Claim 17 (depends on 11)

The computer program product of claim 11, wherein the hardened neural network is hardened against gradient based attacks such that perturbations in input data to the hardened neural network do not cause misclassification by the hardened neural network.

### Claim 18 (depends on 11)

The computer program product of claim 11, wherein the computer readable program further causes the data processing system to:
receive, by the hardened neural network, input data to be processed;
generate, by the hardened neural network, an output vector comprising a plurality of probability values stored in vector slots, wherein each vector slot is associated with a different class in a plurality of predefined classes such that the probability value stored in a vector slot indicates a probability that the input data is properly classified into a class corresponding to the vector slot;
provide, by the hardened neural network, the output vector to a cognitive system; and
execute, by the cognitive system, a cognitive operation based on the probability values stored in the output vector.

### Claim 19

A data processing system comprising:
at least one processor; and
at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to:
configure the data processing system to implement a hardened neural network engine that operates on a neural network to harden the neural network against evasion attacks and generates a hardened neural network;
generate a reference training data set based on an original training data set;
process, by the neural network, the original training data set and the reference training data set to generate first and second output data sets;
calculate, by the hardened neural network engine, a modified loss function of the neural network, wherein the modified loss function is a combination of an original loss function associated with the neural network, and a delta function of t a ratio of a similarity between the original training data set and the reference data set and a similarity between the first output data set and the second output data set; and
train, by the hardened neural network engine, the neural network based on the modified loss function to generate the hardened neural network, wherein the delta function, Delta, is as follows:
Delta(X,X′,Y,Y′)=cos(X,X′)/cos(Y,Y′),
wherein X is the original training data set, X′ is the reference training data set, Y is the first output data set, Y′ is the second output data set, and cos is a cosine similarity function.