Obfuscating inference operations of a machine learning model

A system comprising a hardware device and one or more storage devices storing instructions that when executed by the hardware device cause the hardware device to perform operations, the operations comprising:

• receiving, by the hardware device, data representing a machine learning model comprising a plurality of model parameters for inference operations, wherein the hardware device comprises a set of computation units arranged in one or more processing elements;
• obtaining instructions for performing obfuscating operations configured to obfuscate one or more measurable characteristics of the machine learning model when the machine learning model is executed by the one or more processing elements;
• causing a first portion of the set of computation units to perform the inference operations of the machine learning model;
• and
• causing a second portion of the set of computation units to perform the obfuscating operations concurrently with the first portion of the set of computation units performing the inference operations.

One or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving, by a hardware device, data representing a machine learning model comprising a plurality of model parameters for inference operations, wherein the hardware device comprises a set of computation units arranged in one or more processing elements;
• obtaining instructions for performing obfuscating operations configured to obfuscate one or more measurable characteristics of the machine learning model when the machine learning model is executed by the one or more processing elements;
• causing a first portion of the set of computation units to perform the inference operations of the machine learning model;
• and
• causing a second portion of the set of computation units to perform the obfuscating operations concurrently with the first portion of the set of computation units performing the inference operations.