System and method for securing a neural network runtime engine

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method of securing a neural network (NN) runtime engine, the method comprising:

• obtaining a NN model and a set of weights for the NN model;
• generating first program code to perform inference operations, wherein generating the first program code comprises:
• configuring the inference operations to proceed with at least one of adaptive latency or adaptive accuracy;
• wherein adaptive latency dynamically modifies latency of the inference operations during execution of the NN runtime engine in response to input to the NN runtime;
• and
• wherein adaptive accuracy dynamically modifies accuracy of the inference operations during execution of the NN runtime engine in response to input to the NN runtime engine
• wherein the at least one of adaptive latency or adaptive accuracy comprise:
• processing decoy operators to resist reverse engineering of model performance and accuracy;
• generating second program code to provide runtime services comprising one or more security elements to secure the NN model;
• integrating the first program code and the second program code into the NN runtime engine;
• and
• training the NN model with the use of a loss function comprising a first term for determining an accuracy of the NN model and second term for determining a quality and a robustness of a digital watermark embedded in the NN model, wherein the one or more digital watermarks are tuned during training of the NN model for the quality and the robustness of the digital watermark embedded in the NN model, wherein the quality of the digital watermark of the NN model is based upon the length of the digital watermark and the number of unique symbols of the digital watermark and the robustness of the digital watermark of the NN model is based upon a number of bits used for the digital watermark.

A system for securing a neural network (NN) runtime engine, comprising:

• at least one processor;
• and
• memory storing instructions that, when executed by the at least one processor, cause the system to:
• obtain a NN model and a set of weights for the NN model;
• generate first program code to perform inference operations, wherein generating the first program code comprises:
• configuring the inference operations to proceed with at least one of adaptive latency or adaptive accuracy;
• wherein adaptive latency dynamically modifies latency of the inference operations during execution of the NN runtime engine in response to input to the NN runtime;
• and
• wherein adaptive accuracy dynamically modifies accuracy of the inference operations during execution of the NN runtime engine in response to input to the NN runtime engine
• wherein the at least one of adaptive latency or adaptive accuracy comprise:
• processing decoy operators to resist reverse engineering of model performance and accuracy;
• generate second program code to provide runtime services comprising one or more security elements to secure the NN model;
• integrate the first program code and the second program code into the NN runtime engine;
• and
• train the NN model with the use of a loss function comprising a first term for determining an accuracy of the NN model and second term for determining a quality and a robustness of a digital watermark embedded in the NN model, wherein the one or more digital watermarks are tuned during training of the NN model for the quality and the robustness of the digital watermark embedded in the NN model, wherein the quality of the digital watermark of the NN model is based upon the length of the digital watermark and the number of unique symbols of the digital watermark and the robustness of the digital watermark of the NN model is based upon a number of bits used for the digital watermark.