Optimization of neural network in equivalent class space

A device, comprising:

• a processing unit;
• and
• a memory coupled to the processing unit and comprising instructions stored thereon, the instructions, when executed by the processing unit, causing the device to perform acts comprising:
• determining a plurality of base paths running through a plurality of layers of a learning network, each node in the plurality of layers utilizing an activation function with a scaling invariant property to process an input from a node of a previous layer, each base path comprising a single node in each of the plurality of layers, and input processing in the plurality of base paths being linearly independent from each other;
• for each of the plurality of base paths, determining a combined value of respective parameters associated with the nodes in the base path resulting in combined values of the parameters for the plurality of base paths, a parameter of the respective parameters associated with a node of the nodes in each base path being for adjusting the input obtained by the node from a node of a previous layer;
• and
• updating, for the plurality of base paths, values of the parameters associated with the nodes in the plurality of base paths based on the combined values.

A computer program product being stored in a non-transitory computer storage medium and comprising machine executable instructions which, when executed by a device, causing the device to:

• determine a plurality of base paths running through a plurality of layers of a learning network, each node in the plurality of layers utilizing an activation function with a scaling invariant property to process input from a node of a previous layer, each base path comprising a single node in each of the plurality of layers, and input processing in the plurality of base paths being linearly independent from each other;
• for each of the plurality of base paths, determine a combined value of respective parameters associated with the nodes in the base path resulting in combined values of the parameters for the plurality of base paths, a parameter of the respective parameters associated with a node of the nodes in each base path being for adjusting the input obtained by the node from a node of a previous layer;
• and
• update, for the plurality of base paths, values of the parameters associated with the nodes in the plurality of base paths based on the combined values.