System and method for tiny machine learning using block floating point
AssigneeHong Kong Applied Science and Technology Research Institute Company Limited
InventorsChi Wai NG, Suk Ling LI, Pei Fung LAM
Life of this patent
Estimating the term…
Abstract
A system includes a first FP-to-BFP converter, a second FP-to-BFP converter, an 8-bit integer multiplier, an adder, an accumulator, and a BFP-to-FP converter. The first and second FP-to-BFP converters receive 32-bit floating-point pixel and filter data, respectively, reducing their mantissas to 8-bit BFP format. The 8-bit integer multiplier processes these BFP values via multiply-accumulate operations, generating a 16-bit product. The adder accumulates multiple 16-bit products into a 64-bit sum, which the accumulator further aggregates. The BFP-to-FP converter transforms the 64-bit accumulated sum into a 32-bit floating-point output.
Claims20
A system for executing deep learning model computations, comprising:
- a first floating-point to block floating-point (FP-to-BFP) converter configured to receive pixel data in a 32-bit floating-point format and convert the pixel data by reducing a mantissa of the pixel data to 8 bits, thereby obtaining a block floating-point (BFP) format with an 8-bit mantissa;
- a second FP-to-BFP converter configured to receive filter data in a 32-bit floating-point format and convert the filter data by reducing a mantissa of the filter data to 8 bits, thereby obtaining a BFP format with an 8-bit mantissa;
- an 8-bit integer multiplier configured to receive BFP representations from the first FP-to-BFP converter and the second FP-to-BFP converter and to perform multiply-accumulate operations to generate a 16-bit product;
- an adder configured to receive the 16-bit product from the 8-bit integer multiplier and to accumulate multiple 16-bit products, thereby generating a 64-bit accumulated sum;
- an accumulator configured to receive the 64-bit accumulated sum from the adder and to aggregate accumulated sums;
- and
- a block floating-point to floating-point (BFP-to-FP) converter configured to receive the 64-bit accumulated sum from the accumulator and convert the 64-bit accumulated sum into a 32-bit floating-point output.
A method for converting floating-point data to block floating-point (BFP) format, comprising:
- receiving, by a floating-point to block floating-point (FP-to-BFP) converter, data in a 32-bit floating-point format;
- determining, by the FP-to-BFP converter, a shared exponent for a block of floating-point values as the maximum exponent among all the floating-point values in the block;
- computing, by the FP-to-BFP converter, a right-shift amount for each of the floating-point values based on the difference between its original exponent and the determined shared exponent;
- applying, by the FP-to-BFP converter, a right shift to mantissa of each of the floating-point values;
- and
- truncating, by the FP-to-BFP converter, the 24-bit mantissa of each floating-point value to 8 bits to obtain a BFP format.
A tiny machine learning (ML) platform, comprising:
- a tiny ML operations accelerator configured to execute convolutional neural network (CNN) operations using approximate computing techniques and comprising:
- a first floating-point to block floating-point (FP-to-BFP) converter configured to receive pixel data in a 32-bit floating-point format and convert the pixel data by reducing a mantissa of the pixel data to 8 bits, thereby obtaining a block floating-point (BFP) format with an 8-bit mantissa;
- a second FP-to-BFP converter configured to receive filter data in a 32-bit floating-point format and convert the filter data by reducing a mantissa of the filter data to 8 bits, thereby obtaining a BFP format with an 8-bit mantissa;
- and
- an 8-bit integer multiplier configured to receive BFP representations from the first FP-to-BFP converter and the second FP-to-BFP converter and to perform multiply-accumulate (MAC) operations to generate a 16-bit product;
- a single instruction multiple data multiply-accumulate (SIMD MAC) accelerator coupled to the tiny ML operations accelerator and configured to perform signal feature extraction and enhance vectorized execution of CNN computations;
- a microcontroller unit (MCU) core configured to manage execution control, memory access, and coordination between the tiny ML operations accelerator and the SIMD MAC accelerator;
- a memory coupled to the MCU core, the tiny ML operations accelerator, and the SIMD MAC accelerator;
- and
- a multiplexer (MUX) configured to dynamically route data between the MCU core, the memory, the tiny ML operations accelerator, and the SIMD MAC accelerator.
Dependencies are inferred from the claim text.