# US 12,182,274 B2 · Testing adversarial robustness of systems with limited access

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,274 B2
- Document ID: US12182274B2
- Status: Granted
- Kind: B2
- Application number: 18/382,107
- Filed: 2023-10-20
- Published: 2024-02-08
- Granted: 2024-12-31
- Earliest claimed: 2019-01-24
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Pin-Yu Chen; Sijia Liu; Lingfei Wu; Chia-Yu Chen
- Web page: https://patentgrep.com/patent/US12182274B2

## CPC

- G06F 21/577: {Assessing vulnerabilities and evaluating computer system security}
- G06F 18/2413: based on distances to training or reference patterns
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06F 2221/034: Test or assess a computer or a system
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An adversarial robustness testing method, system, and computer program product include testing, via an accelerator, a robustness of a black-box system under different access settings, where the testing includes tearing down the robustness testing to a subtask of a predetermined size.

## Claims

### Claim 1

A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings;
given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and a legitimate image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the different access settings comprise:
a soft-label setting; and
a hard-label setting.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
using the accelerator and a gradient descent technique, in the soft-label setting, to find the adversarial examples and summarize a first robustness statistic; and
using the accelerator and a smoothing function, for the hard-label setting, to summarize a second robustness statistic.

### Claim 4 (depends on 1)

The method of claim 1, wherein the accelerator comprises a function including an efficient gradient estimation via a random directional estimate and averaging.

### Claim 5 (depends on 1)

The method of claim 1, wherein the accelerator comprises a function including a dimension reduction of an input.

### Claim 6 (depends on 1)

The method of claim 1, wherein the accelerator comprises a function including a problem splitting between a black-box loss function and a white-box adversarial distortion function.

### Claim 7 (depends on 1)

The method of claim 1, embodied in a cloud-computing environment.

### Claim 8

A computer program product for adversarial robustness testing for checking a learning performance of a black-box system, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings;
given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.

### Claim 9 (depends on 8)

The computer program product of claim 8, wherein the different access settings comprise:
a soft-label setting; and
a hard-label setting.

### Claim 10 (depends on 9)

The computer program product of claim 9, further comprising:
using the accelerator and a gradient descent technique, in the soft-label setting, to find the adversarial examples and summarize a first robustness statistic; and
using the accelerator and a smoothing function, for the hard-label setting, to summarize a second robustness statistic.

### Claim 11 (depends on 8)

The computer program product of claim 8, wherein the accelerator comprises a function including an efficient gradient estimation via a random directional estimate and averaging.

### Claim 12 (depends on 8)

The computer program product of claim 8, wherein the accelerator comprises a function including a dimension reduction of an input.

### Claim 13 (depends on 8)

The computer program product of claim 8, wherein the accelerator comprises a function including a problem splitting between a black-box loss function and a white-box adversarial distortion function.

### Claim 14

An adversarial robustness testing system for checking a learning performance of a black-box system, the system comprising:
a processor; and
a memory, the memory storing instructions to cause the processor to perform:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings;
given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.

### Claim 15 (depends on 14)

The system of claim 14, wherein the different access settings comprise:
a soft-label setting; and
a hard-label setting.

### Claim 16 (depends on 15)

The system of claim 15, further comprising:
using the accelerator and a gradient descent technique, in the soft-label setting, to find the adversarial examples and summarize a first robustness statistic; and
using the accelerator and a smoothing function, for the hard-label setting, to summarize a second robustness statistic.

### Claim 17 (depends on 14)

The system of claim 14, wherein the accelerator comprises a function including an efficient gradient estimation via a random directional estimate and averaging.

### Claim 18 (depends on 14)

The system of claim 14, wherein the accelerator comprises a function including a dimension reduction of an input.

### Claim 19 (depends on 14)

The system of claim 14, wherein the accelerator comprises a function including a problem splitting between a black-box loss function and a white-box adversarial distortion function.

### Claim 20

A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
receiving a first classification of an input as an output from the black-box system; and
determining a minimal change to the input such that a second classification is received as the output from the black-box system;
given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using an accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.

### Claim 21

A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
finding a minimal change to an input such that a classification of an output from the black-box system is different than an original classification of the input, wherein the testing includes a function to reduce an attack space in the adversarial attack for query efficiency;
given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using an accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.