Systems, apparatuses, methods, and non-transitory computer-readable storage devices for optimizing artificial neural network

One or more circuits for performing actions comprising:

• generating an input computational graph (CG) for representing a neural-network architecture, the input CG comprising one or more nodes each corresponding to an operation;
• and
• performing a plurality of optimization steps for at least one round to obtain a modified neural-network architecture represented by a modified CG for obtaining a neural network for use in one or more computing devices;
• wherein the plurality of optimization steps comprise:
• extracting a plurality of subgraphs from the input CG,
• obtaining the modified CG by replacing one or more of the plurality of subgraphs with one or more replacement subgraphs, respectively,
• evaluating one or more metrics of a modified neural-network architecture represented by the modified CG, and
• based on the evaluation, selecting the modified CG or the input CG;
• and
• wherein, when the plurality of optimization steps are performed for a plurality of rounds,
• the selected CG obtained in one of the plurality of rounds is used as the input CG for a next one of the plurality of rounds.

One or more non-transitory computer-readable storage devices comprising computer-executable instructions, wherein the instructions, when executed, cause one or more circuits to perform actions comprising:

• generating an input computational graph (CG) for representing a neural-network architecture, the input CG comprising one or more nodes each corresponding to an operation;
• and
• performing a plurality of optimization steps for at least one round to obtain a modified neural-network architecture represented by a modified CG for obtaining a neural network for use in one or more computing devices;
• wherein the plurality of optimization steps comprise:
• extracting a plurality of subgraphs from the input CG,
• obtaining the modified CG by replacing one or more of the plurality of subgraphs with one or more replacement subgraphs, respectively,
• evaluating one or more metrics of a modified neural-network architecture represented by the modified CG, and
• based on the evaluation, selecting the modified CG or the input CG;
• and
• wherein, when the plurality of optimization steps are performed for a plurality of rounds,
• the selected CG obtained in one of the plurality of rounds is used as the input CG for a next one of the plurality of rounds.