Method for building a resource-frugal neural network

A method for building a neural network configured to be run on a destination processing unit, the method being implemented by a system comprising a computer and a memory storing a learning dataset,

• the method comprising:
• providing a neural network having an initial topology comprising an input layer, a hidden layer, and a set of output nodes, and training said initial topology over the learning dataset, and
• optimizing the topology of the neural network,
• wherein optimizing the topology of the neural network comprises at least one iteration of the following steps:
• for each of a plurality of candidate topological changes, estimating a variation induced by the candidate topological change on:
• the neural network's error, and
• a value of at least one physical quantity needed for executing the neural network on the destination processing unit,
• selecting at least one of the candidate topological changes based on said estimation and updating the topology of the neural network according to said selected topological change, and
• training the updated neural network over the learning dataset,
• wherein the estimation step for a given candidate topological change comprises estimating the variation induced by the candidate topological change on a cost function formed as a weighted sum of:
• a term representative of a best error of the neural network obtainable by the neural network incorporating the candidate topological change on the training dataset, and
• at least one penalty term corresponding to each respective physical quantity exceeding a corresponding maximum value,
• wherein the penalty term corresponding to a physical quantity is a positive part of a difference between a value of the physical quantity needed for executing the neural network incorporating the candidate topological change and a corresponding maximum value.

A method for building a neural network configured to be run on a destination processing unit, the method being implemented by a system comprising a computer and a memory storing a learning dataset,

• the method comprising:
• providing a neural network having an initial topology comprising an input layer, a hidden layer, and a set of output nodes, and training said initial topology over the learning dataset, and
• optimizing the topology of the neural network,
• wherein optimizing the topology of the neural network comprises at least one iteration of the following steps:
• for each of a plurality of candidate topological changes, estimating a variation induced by the candidate topological change on:
• the neural network's error, and
• a value of at least one physical quantity needed for executing the neural network on the destination processing unit,
• selecting at least one of the candidate topological changes based on said estimation and updating the topology of the neural network according to said selected topological change, and
• training the updated neural network over the learning dataset,
• wherein estimating the variation induced by the candidate topological change on the neural network's error comprises estimating a best error obtainable by the neural network incorporating the candidate topological change on the training dataset, by computing the Lagrangian of the neural network including the candidate topological change.