Depth-constrained knowledge distillation for inference on encrypted data

An apparatus, comprising:

• a processor;
• computer memory holding computer program instructions executed by the processor, the computer program instructions configured to create a full homomorphic encryption (FHE)-compatible machine learning model, the computer program instructions configured to:
• obtain a first machine learning model, the first machine learning model having been pre-trained using first training data;
• receive an input data distribution, and a set of FHE constraints;
• and
• construct a second machine learning model that satisfies the set FHE constraints, the second machine learning model constructed by:
• using the input data distribution to generate a transfer data set;
• and
• learning parameters of the second machine learning model using, as synthetic training data, predictions of the first machine learning model on the transfer data set.

A computer program product in a non-transitory computer readable medium for use in a data processing system to create a full homomorphic encryption (FHE)-compatible machine learning model, the computer program product holding computer program instructions that, when executed by the data processing system:

• obtain a first machine learning model, the first machine learning model having been pre-trained using first training data;
• receive an input data distribution, and a set of FHE constraints;
• and
• construct a second machine learning model that satisfies the set FHE constraints, the second machine learning model constructed by:
• using the input data distribution to generate a transfer data set;
• and
• learning parameters of the second machine learning model using, as synthetic training data, predictions of the first machine learning model on the transfer data set.