Dynamic hardware aware model partitioning

A method implemented in a data processing system for partitioning an artificial intelligence model, the method comprising:

• analyzing program code representing a neural network architecture of an artificial intelligence model to generate a map of layers of the neural network architecture by identifying connections between layers of the neural network architecture and inspecting a flow of information between the layers of the artificial intelligence model, the map including connectivity information representing the connections between the layers of the neural network architecture that facilitate information flowing between the layers, and dependency information representing relationships between layers in which an output of one layer is provided as an input to another layer;
• obtaining hardware information indicating computing hardware components available for operating portions of the artificial intelligence model, the computing hardware components being associated with one of a plurality of hardware security domains providing security features for protecting sensitive data being processed by the computing hardware components from unauthorized access, the security features comprising physical components, software, or a combination thereof that prevent unauthorized access or tampering with the sensitive data;
• determining security requirements for each of the layers of the neural network architecture based on the map of the layers of the neural network architecture by analyzing the map to determine a position of each of the layers relative to an input layer of the neural network architecture and a layer type associated with each of the layers;
• determining which hardware components can satisfy the security requirements for each of the layers of the neural network architecture;
• partitioning the neural network architecture into a partitioned neural network architecture that includes partitions comprising one or more layers of the neural network architecture based on the security requirements of each of the layers to be operated on selected hardware components from among the computing hardware components, the selected hardware components satisfy the security requirements of the layers included in a respective partition;
• and
• operating the partitioned neural network architecture on the selected hardware components.

A machine-readable medium on which are stored instructions that, when executed, cause a processor of alone or in combination with other processors to perform operations of:

• analyzing program code representing a neural network architecture of an artificial intelligence model to generate a map of layers of the neural network architecture by identifying connections between layers of the neural network architecture and inspecting a flow of information between the layers of the artificial intelligence model, the map including connectivity information representing the connections between the layers of the neural network architecture that facilitate information flowing between the layers, and dependency information representing relationships between layers in which an output of one layer is provided as an input to another layer;
• obtaining hardware information indicating computing hardware components available for operating portions of the artificial intelligence model, the computing hardware components being associated with one of a plurality of hardware security domains providing security features for protecting sensitive data being processed by the computing hardware components from unauthorized access, the security features comprising physical components, software, or a combination thereof that prevent unauthorized access or tampering with the sensitive data;
• determining security requirements for each of the layers of the neural network architecture based on the map of the layers of the neural network architecture by analyzing the map to determine a position of each of the layers relative to an input layer of the neural network architecture and a layer type associated with each of the layers;
• determining which hardware components can satisfy the security requirements for each of the layers of the neural network architecture;
• partitioning the neural network architecture into a partitioned neural network architecture that includes partitions comprising one or more layers of the neural network architecture based on the security requirements of each of the layers to be operated on selected hardware components from among the computing hardware components, the selected hardware components satisfy the security requirements of the layers included in a respective partition;
• and
• operating the partitioned neural network architecture on the selected hardware components.