Artificial intelligence adversarial vulnerability audit tool
AssigneeCalypso Ai Corp
InventorsVictor Ardulov, Neil Serebryany, Tyler Sweatt, David Gibian
Life of this patent
Estimating the term…
Abstract
An image with a known, first classification by the machine learning model is received. This image is then iteratively modified using at least one perturbation algorithm and such modified images are input into the machine learning model until such time as the machine learning model outputs a second classification different from the first classification. Data characterizing the modifications to the image that resulted in the second classification can be provided (e.g., displayed in a GUI, loaded into memory, stored in physical persistence, transmitted to a remote computing device). Related apparatus, systems, techniques and articles are also described.
Claims14
A computer-implemented method for characterizing the robustness of a computer-vision machine learning model comprising:
- receiving an image with a known, first classification by the machine learning model;
- iteratively (i) modifying the image using at least one perturbation algorithm, wherein the iterative modifying of the image simulates an attack on the machine learning model and wherein the attack simulates a modification of a physical object and (ii) inputting the corresponding modified image into the machine learning model until such time that the machine learning model outputs a second classification different from the first classification;
- providing data characterizing the modifications to the image that results in the machine learning model outputting the second classification;
- and
- printing an overlay comprising the modifications, wherein the overlay is printed on a sticker which can be affixed to or positioned in relation to the physical object.
A system for characterizing the robustness of a computer-vision machine learning model comprising:
- at least one data processor;
- and
- memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
- receiving an image with a known, first classification by the machine learning model;
- continually modifying the image using at least one perturbation algorithm wherein continually modifying of the image simulates an attack on the machine learning model and wherein the attack simulates a modification of a physical object and inputting the corresponding modified image into the machine learning model until such time that the machine learning model outputs a second classification different from the first classification;
- providing data characterizing the modifications to the image that resulted in the machine learning model outputting the second classification;
- and
- printing an overlay comprising the modifications, wherein the overlay is printed on a sticker which can be affixed to or positioned in relation to the physical object.
A non-transitory computer program product storing instructions which, when executed by at least one computing device, implement operations comprising:
- receiving an image with a known, first classification by the machine learning model;
- iteratively modifying the image using at least two perturbation algorithms including an algorithm that simulates an attack on the machine learning model and wherein the attack simulates a modification of a physical object and inputting the corresponding modified image into the machine learning model until such time that the machine learning model outputs a desired, second classification different from the first classification;
- providing data characterizing the modifications to the image that resulted in the machine learning model outputting the second classification;
- and
- printing an overlay comprising the modifications, wherein the overlay is printed on a sticker which can be affixed to or positioned in relation to the physical object.
Dependencies are inferred from the claim text.