Hardware and neural architecture co-search

A method comprising:

• obtaining a specification of a function and a plurality of hardware design parameters, the hardware design parameters including a memory capacity, a number of computational resources, a communication bandwidth, and a template configuration for performing neural architecture inference;
• determining, for each neural architecture among a plurality of neural architectures, each neural architecture having been trained to perform the function with an accuracy, an overall latency of performance of inference of the neural architecture by an accelerator within the hardware design parameters;
• and
• selecting, from among the plurality of neural architectures, a neural architecture based on the overall latency and the accuracy;
• wherein the determining the overall latency further includes determining a latency model of a plurality of hyper-parameters to execute the performance of inference of the neural architecture by the accelerator while interfacing with an external memory storing activation data.

An apparatus comprising:

• an obtaining section configured to obtain a specification of a function and a plurality of hardware design parameters, the hardware design parameters including a memory capacity, a number of computational resources, a communication bandwidth, and a template configuration for performing neural architecture inference;
• a determining section configured to determine, for each neural architecture among a plurality of neural architectures, each neural architecture having been trained to perform the function with an accuracy, an overall latency of performance of inference of the neural architecture by an accelerator within the hardware design parameters;
• and
• a selecting section configured to select, from among the plurality of neural architectures, a neural architecture based on the overall latency and the accuracy;
• wherein the determining section is further configured to determine the overall latency by determining a latency model of a plurality of hyper-parameters to execute the performance of inference of the neural architecture by the accelerator while interfacing with an external memory storing activation data.