Artificial neural network and method of controlling fixed point in the same

An artificial neural network (ANN) system, comprising:

• a virtual overflow detection circuit configured to:
• generate a plurality of thermometer codes corresponding to a plurality of quantization values, the plurality of quantization values being obtained by performing a quantization operation based on a fixed-point format,
• generate virtual overflow information indicating a frequency distribution of valid bit numbers of the plurality of quantization values based on the generated plurality of thermometer codes, wherein a valid bit number indicates a non-zero most significant bit of an integer portion of a quantized value;
• and
• a data format controller configured to update the fixed-point format based on the generated virtual overflow information,
• wherein the virtual overflow detection circuit comprises a thermometer code generation circuit configured to generate the plurality of thermometer codes, an accumulation circuit configured to generate a plurality of accumulation values, and a subtraction circuit configured to provide a generated plurality of count signals as the virtual overflow information.

A method of controlling a fixed point format in an artificial neural network (ANN) system, the method comprising:

• performing node operations with respect to a plurality of nodes included in each layer of an ANN to obtain a plurality of result values of the node operations;
• performing a quantization operation on the obtained plurality of result values based on a k-th fixed-point format for a current quantization of the each layer to obtain a plurality of quantization values;
• generating, by a virtual overflow detection circuit, virtual overflow information indicating a frequency distribution of valid bit numbers of the obtained plurality of quantization values, wherein a valid bit number indicates a non-zero most significant bit of an integer portion of a quantized value;
• and
• determining a (k+1)-th fixed-point format for a next quantization of the each layer based on the generated virtual overflow information,
• wherein the generating of the virtual overflow information comprises:
• generating a plurality of thermometer codes corresponding to the obtained plurality of quantization values;
• generating a plurality of count signals indicating frequency values of the valid bit numbers based on the generated plurality of thermometer codes;
• and
• providing the generated plurality of count signals as the virtual overflow information.