Detection of Privacy Attacks on Machine Learning Models

A computer-implemented method comprising:

• inputting each of a plurality of queries into an artificial intelligence (AI) model comprising a plurality of layers including an input layer, an output layer, and at least one intermediate layer between the input layer and the output layer, each intermediate layer, during inference, outputting a plurality of activations;
• intercepting, for each query, activations from at least one of the intermediate layers;
• generating, using the intercepted activations and reference activations, a first metric characterizing a measure of similarity among curves;
• generating, using the intercepted activations and the reference activations, a second metric based on a class-conditioned distance;
• and
• initiating a remediation action to protect the AI model from a privacy attack when the first metric is above a first threshold and the second metric is above a second threshold.

A computer-implemented method comprising:

• intercepting each of a plurality of queries of an artificial intelligence (AI) model;
• inputting the intercepted queries into a surrogate model corresponding to the AI model, the surrogate model comprising a plurality of layers including an input layer, an output layer, and at least one intermediate layer between the input layer and the output layer, each intermediate layer, during inference, outputting a plurality of surrogate activations, the surrogate model being a more compact representation of the AI model;
• intercepting, for each query, surrogate activations from at least one of the intermediate layers;
• determining whether a distribution of the intercepted surrogate activations across the queries indicates that the queries seek to cause the AI model to behave in an undesired manner by conducting a distance-based similarity analysis between the intercepted surrogate activations and reference surrogate activations;
• allowing queries to be input into the AI model in response to a determination that such queries do not seek to cause the AI Model to behave in an undesired manner;
• and
• initiating remediation actions preventing queries from being input into the AI model in response to a determination that such queries seeks to cause the AI model to behave in an undesired manner.