Partial activation of multiple pathways in neural networks

A system for approximating a neural network, the system comprising:

• one or more memories to store a neural network comprising N synapses or filters;
• one or more processors configured to:
• partially-activate the neural network by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer, wherein the M partial pathways cumulatively span only a subset of the N synapses or filters of the neural network such that a plurality of the remaining the N synapses or filters are not computed;
• aggregate the M partial outputs of the plurality of M partial pathways to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network;
• and
• perform training or prediction of the neural network based on the aggregated output.