Method for watermarking a machine learning model

#This application is related to U.S. patent application Ser. No. 16/779,737, filed on Feb. 3, 2020 by Michiels, and entitled “Method for Watermarking a Machine Learning Model.”

#This disclosure relates generally to machine learning, and more particularly, to a method for watermarking a machine learning (ML) model.

#Machine learning (ML) is becoming more widely used in many of today's applications, such as applications involving forecasting and classification. Generally, a ML model is trained, at least partly, before it is used. Training data is used for training a ML model. Machine learning models may be classified by how they are trained. Supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning are examples of training techniques. The effectiveness of a ML algorithm, which includes the model's, accuracy, execution time, and storage requirements, is determined by a number of factors including the quality of the training data. The expertise, time, and cost that goes into the compilation of a representative training set can make the training data set as well as the model obtained using the training data set very valuable assets.

#The present invention is illustrated by way of example and is not limited by the accompanying figures, in which like references indicate similar elements. Elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale.

#FIG. 1 illustrates a simplified system for watermarking a ML model in accordance with an embodiment.

#FIG. 2 illustrates an example image useful for training an object detector.

#Generally, there is provided, a method for watermarking a ML model. The watermark is generated by modifying a subset of labeled training data used for performing supervised training of a ML model to be watermarked. For discussion purposes, the ML model is based on a neural network (NN) algorithm and the training data includes a plurality of images. In other embodiments, the ML algorithm and training data may be different. In the method, a subset of images of the labeled training data is selected. In one embodiment, the selected subset includes objects of a predetermined classification, for example, boats, or cars. A pixel pattern is selected to combine with each image of the subset of labeled training data. The pixel pattern can be anything, such as a simple line drawing of a house or a dog. A bounding box is drawn around each of the objects of the predetermined class in the subset of images. The pixel pattern is overlaid or superimposed over the objects in the subset of labeled training data. The subset of labeled training data is then relabeled to be different from the original problem domain label. For example, if the subset is chosen to be all a certain class of objects, for example, buses, then the objects with the overlaid pixel pattern are relabeled to be something distinctly different from the predetermined class. In one embodiment, the pixel pattern is scaled for each bounding box for each of the objects such that a first dimension of the pixel pattern substantially matches a first dimension of the bounding box. Further, shape or scale of the image may be changed without regard to an aspect ratio such that a second dimension of the pixel pattern matches a second dimension of the bounding box. Then the relabeled subset is included with the labeled training data and the ML model is trained with the labeled training data and the relabeled subset. In one embodiment, a transparency, or opacity, of the pixel pattern may be varied. Further, the pixel pattern may include a first set of pixels having a first opacity factor and a second set of pixels having a second opacity factor, where the first opacity factor and the second opacity factor are varied independently. In one embodiment, the pixel pattern may also be transformed or altered using one or more of rotating, scaling, mirroring, or the like. In another embodiment, the bounding box and the label may be removed from around the object but leaving the pixel pattern over the object. Without the bounding box and label, the ML model is trained that the object with the overlaid pixel pattern is part of the background and will return no output.

#A trained ML model, either for classification, or for object detection, is thus watermarked and may be used for inference operation in any of a number of applications that require the detection of objects in images, such as an autonomously driven automobile. To detect whether a ML model is the ML model with the watermark, another subset of images is created. The another subset of test samples may be selected from the labeled set of ML training data or sourced elsewhere as long as the images of the subset are of the same class as the subset used for watermark training. In the case of an object detector, the same pixel pattern is inserted, or overlaid, into the bounding boxes of a sample of the subset to be used for watermark detection. Likewise, in a ML model used as a classifier, the pixel pattern overlays the image. Then, during inference operation, the detection subset of samples is presented to the ML model. The images or objects in the test subset do not have to be labeled. If the ML model provides output classifications in response to the detection subset consistent with the training of the ML model, then the ML model is likely to be the watermarked model. The detection subset of samples may be kept secret.

#The method for watermarking a ML model may be used in, for example, classifiers and object detectors and provides a reliable method of watermarking that can be used without having full access to the ML model. Also, the samples chosen for the watermark are samples of the problem domain of the ML model. Using problem domain samples improves the reliability of the watermark because it lessens the possibility that the watermark prediction output is a prediction outside of the problem domain for which the ML model was trained. Using two opacities for different parts of a pixel pattern provides at least two benefits: First, the less transparent part of the pixel pattern is easier for the ML model to learn, and second, the more transparent part makes the object in the image easier to see through the pixel pattern, and thus the watermark may have more credibility.