Systems and Methods for Efficient Differentiable Logic Gate Networks

A computer-implemented method for training a neural network, comprising:

• providing a neural network comprising a plurality of computational nodes, wherein at least one computational node comprises a hyper-logic gate, and wherein the hyper-logic gate, during training, is realized by a differentiable k-input lookup table, where k is an integer greater than or equal to 1, the differentiable k-input lookup table having:
• (i) k data inputs, each receiving a real value;
• and
• (ii) N configuration inputs, where N is an integer greater than k, each configuration input receiving a real value;
• computing an output of the at least one computational node as a differentiable function of the k data inputs and the N configuration inputs, wherein, when the k data inputs and the N configuration inputs are restricted to binary values, the output equals the configuration input indexed by the binary values of the k data inputs, wherein at least one of the N configuration inputs is provided by an output of one of the computational nodes in the neural network;
• computing a loss from outputs of the neural network;
• updating parameters of the neural network based on gradients or approximate gradients of the loss propagated through the differentiable k-input lookup table;
• and
• discretizing the neural network, after training, such that the inputs to the hyper-logic gate are binary values, and the output of the hyper-logic gate equals the configuration input indexed by the binary values of the k data inputs.

An application-specific integrated circuit (ASIC), comprising:

• logic circuitry implementing a fixed logic gate network,
• wherein the fixed logic gate network is generated by a process comprising:
• providing a neural network comprising a plurality of computational nodes, wherein at least one computational node comprises a hyper-logic gate, and wherein the hyper-logic gate, during training, is realized by a differentiable k-input lookup table, k≥1, the differentiable k-input lookup table having:
• (i) k data inputs, each receiving a real value;
• and
• (ii) N configuration inputs, where N is an integer greater than k, each configuration input receiving a real value;
• computing an output of the at least one computational node as a differentiable function of the k data inputs and the N configuration inputs, wherein, when the k data inputs and the N configuration inputs are restricted to binary values, the output equals the configuration input indexed by the binary values of the k data inputs, and wherein at least one of the N configuration inputs is provided by an output of one of the plurality of computational nodes in the neural network;
• computing a loss from outputs of the neural network;
• updating parameters of the neural network based on gradients or approximate gradients of the loss propagated through the differentiable k-input lookup table;
• and
• discretizing the neural network, after training, to produce the fixed logic gate network such that inputs to the hyper-logic gate are binary values and the output of the hyper-logic gate equals the configuration input indexed by the binary values of the k data inputs.