Self-supervised data obfuscation in foundation models

A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:

• obtaining, by a computer system, a access to a machine learning model, wherein the machine learning model is a trained machine learning model, wherein the machine learning model is a foundation model, and wherein the machine learning model generates embeddings based on input records;
• obtaining, by the computer system, a training data set;
• determining, by the computer system, an obfuscation transform based on the machine learning model and the training data set by self-supervision, wherein determining the obfuscation transform comprises:
• obtaining training embeddings based on the training set from the machine learning model;
• generating the obfuscation transform based on dimensionality of the training embeddings, wherein the obfuscation transform injects noise by, for each of at least a plurality of dimensions of the training embedding, sampling from a parametric noise distribution and applying the sampled parametric noise to a given embedding;
• training the obfuscation transform by adjusting parameters of the parametric noise distribution according to an objective function that is differentiable, wherein the object function comprises a measure of mutual information and a measure of data loss;
• and
• storing, with the computer system, the trained obfuscation transform in memory.

A method comprising:

• obtaining, with a computer system, a access to machine learning model, wherein the machine learning model is a trained machine learning model, wherein the machine learning model is a foundation model, and wherein the machine learning model generates embeddings based on input records;
• obtaining, with the computer system, a training data set, the training set containing sensitive data;
• determining, with the computer system, an obfuscation transform based on the machine learning model and the training data set, wherein determining the obfuscation transform comprises:
• obtaining training embeddings based on the training set from the machine learning model;
• generating the obfuscation transform based on dimensionality of the training embeddings, wherein the obfuscation transform injects noise by, for each of at least a plurality of dimensions of the training embedding, sampling from a parametric noise distribution and applying the sampled parametric noise to a given embedding;
• training the obfuscation transform by adjusting parameters of the parametric noise distribution according to an objective function that is differentiable, wherein the object function comprises a measure of mutual information and a measure of data loss;
• and
• storing, with the computer system, the obfuscation transform in memory.

A method comprising:

• obtaining, by a computer system, a training dataset of images;
• training, with the computer system, a machine learning models as an multi-layer autoencoder to generate as output a reconstruction of an image in the training dataset based on an input of the image in the training dataset, wherein the autoencoder comprises at least one deterministic layer and wherein training is based on optimization of a value indicative of reconstruction loss and wherein the image is multi-dimensional;
• adding, with the computer system, a stochastic noise layer to the autoencoder layer corresponding to a encoded representation of the image within trained layers of the multi-layer autoencoder, the stochastic noise layer having substantially the same dimensionality as the encoded representation of the image, each of at least a plurality of dimensions of the stochastic noise layer comprising a parametric distribution having a mean and a dispersion,
• wherein the stochastic noise layer injects noise to the encoded representation of the image at each of the at least the plurality of dimensions of the encoded representation of the image by sampling a noise value from the corresponding parametric noise distribution and convolving the sampled noise value with a value of the encoded representation for a respective dimension, and
• wherein the parametric noise distributions for each of the at least the plurality of dimensions are independent;
• adjusting, with the computer system, the mean and deviation of the parametric noise distributions for each of the at least the plurality of dimensions of the stochastic noise layer according to an objective function that is differentiable, wherein the objective function comprises a measure of adversarial loss, an adjustable noise tuning parameter, and a reconstruction loss until a termination criterion is satisfied;
• storing, with the computer system, the autoencoder with the stochastic noise layer comprising the adjusted parametric noise distributions in memory;
• obtaining, by the computer system, one or more images in a production data set;
• generating, with the stored autoencoder with the stochastic noise layers, one or more obfuscated image based on the one or more images in the production data set;
• and
• transmitting, by the computer system, the one or more obfuscated images by an untrusted network to an inference model;
• and
• obtaining, by the computer system, an output from the inference model based on the one or more obfuscated images.