Testing adversarial robustness of systems with limited access
AssigneeInternational Business Machines Corporation
InventorsPin-Yu Chen, Sijia Liu, Lingfei Wu, Chia-Yu Chen
Life of this patent
Estimating the term…
Abstract
An adversarial robustness testing method, system, and computer program product include testing, via an accelerator, a robustness of a black-box system under different access settings, where the testing includes tearing down the robustness testing to a subtask of a predetermined size.
Claims21
A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
- testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings;
- given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
- wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and a legitimate image.
A computer program product for adversarial robustness testing for checking a learning performance of a black-box system, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
- testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings;
- given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
- wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.
An adversarial robustness testing system for checking a learning performance of a black-box system, the system comprising:
- a processor;
- and
- a memory, the memory storing instructions to cause the processor to perform:
- testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings;
- given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
- wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.
A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
- testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
- receiving a first classification of an input as an output from the black-box system;
- and
- determining a minimal change to the input such that a second classification is received as the output from the black-box system;
- given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using an accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
- wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.
A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
- testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
- finding a minimal change to an input such that a classification of an output from the black-box system is different than an original classification of the input, wherein the testing includes a function to reduce an attack space in the adversarial attack for query efficiency;
- given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using an accelerator such that a perturbed example is misclassified to a target class including an incorrect class label by a deep neural network model trained on the legitimate inputs,
- wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between adversarial examples and the legitimate image.
Dependencies are inferred from the claim text.