Back to results
US 12,039,432 B2Granted
Artificial neural network activation function
AssigneeInfineon Technologies AG
InventorAndrew Stevens
Life of this patent
Estimating the term…
Abstract
An artificial neural network (ANN) apparatus can include processing component circuitry that receives linear inputs, and removes linearity from the one or more linear inputs based on an S-shaped saturating activation function that generates a continuous non-linear output. The neurons of the ANN comprise digital bit-wise components configured to transform the linear inputs into the continuous non-linear output.
Claims20
An apparatus for an artificial neural network (ANN) comprising:
- processing circuitry comprising a neuron configured to:
- receive one or more linear inputs;
- compute an S-shaped saturating activation function without a multiplier;
- and
- remove linearity from the one or more linear inputs based on the S-shaped saturating activation function to generate a continuous non-linear output;
- wherein the neuron comprises:
- a multi-bit input terminal configured to receive the one or more linear inputs;
- an absolute value circuit configured to provide a multi-bit result representing an absolute value of the one or more linear inputs;
- a shifter circuit configured to bit-shift the multi-bit result provided by the absolute value circuit;
- and
- digital bit-wise components downstream of the shifter circuit, wherein the absolute value circuit, the shifter circuit, and the digital bit-wise components are collectively configured to transform the one or more linear inputs on the multi-bit input terminal into the continuous non-linear output.
8 more elements
A system to be employed in an artificial neural network (ANN) comprising:
- processing circuitry configured to:
- recursively obtain a plurality of inputs based on extracted data of one or more objects;
- and
- process the plurality of inputs, via one or more neurons, and provide a continuous non-linear output to perform a classification operation or a detection operation, wherein the one or more neurons are configured to modify the plurality of inputs according to an activation function that comprises an S-shaped saturating activation function computed using digital bit-wise operations to generate the continuous non-linear output, and compute the S-shaped saturating activation function without a multiplier;
- wherein a neuron of the one or more neurons comprises:
- a multi-bit input terminal configured to receive the plurality of inputs;
- an absolute value circuit configured to provide a multi-bit result based on the plurality of inputs, the multi-bit result representing an absolute value of the plurality of inputs;
- a bit-wise shifter circuit configured to bit-shift the multi-bit result provided by the absolute value circuit, thereby providing a bit-shifted result;
- and
- digital bit-wise components downstream of the bit-wise shifter circuit and configured to transform bit-shifted result into the continuous non-linear output.
7 more elements
An artificial neural network (ANN) comprising one or more neurons, a neuron comprising:
- a multi-bit neuron input including multiple bitlines;
- an absolute value circuit having an input and an output, the input of the absolute value circuit coupled to the multi-bit neuron input;
- a shift operator circuit having a data input, a data output, and a shift control input, the data input and shift control input each coupled to the output of the absolute value circuit;
- a bit-wise operator circuit having an input and an output, the input of the bit-wise operator circuit coupled to the data output of the shift operator circuit;
- and
- a multiplexer circuit having a first multiplexer input, a second multiplexer input, a multiplexer output, and a multiplexer control terminal, the first multiplexer input and the second multiplexer input coupled to the output of the bit-wise operator circuit, and multiplexer control terminal coupled to at least one bitline of the multi-bit neuron input.
3 more elements
Dependencies are inferred from the claim text.