System and method for synthesis of compact and accurate neural networks (scann)

A system for generating a compact and accurate neural network for a dataset, the system comprising one or more processors configured to:

• provide an initial neural network architecture;
• perform a dataset modification on the dataset, the dataset modification comprising reducing dimensionality of the dataset;
• perform a first compression step on the initial neural network architecture that results in a compressed neural network architecture, the first compression step comprising reducing a number of neurons in one or more layers of the initial neural network architecture based on a feature compression ratio determined by the reduced dimensionality of the dataset;
• and
• perform a second compression step on the compressed neural network architecture, the second compression step comprising iteratively growing at least one of connections and neurons to increase a size of the compressed neural network architecture and iteratively pruning connections to decrease the size of the compressed neural network architecture until a desired neural network architecture has been generated.

A non-transitory computer-readable medium having stored thereon a computer program for execution by a processor configured to perform a method for generating a compact and accurate neural network for a dataset, the method comprising:

• providing an initial neural network architecture;
• performing a dataset modification on the dataset, the dataset modification comprising reducing dimensionality of the dataset;
• performing a first compression step on the initial neural network architecture that results in a compressed neural network architecture, the first compression step comprising reducing a number of neurons in one or more layers of the initial neural network architecture based on a feature compression ratio determined by the reduced dimensionality of the dataset;
• performing a second compression step on the compressed neural network architecture, the second compression step comprising iteratively growing at least one of connections and neurons to increase a size of the compressed neural network architecture and iteratively pruning connections to decrease the size of the compressed neural network architecture until a desired neural network architecture has been generated.