System and method for int9 quantization
AssigneeMarvell Asia Pte, Ltd. (Registration No. 199702379M)
InventorsAvinash Sodani, Ulf Hanebutte, Chia-Hsin Chen
Life of this patent
Estimating the term…
Loading patent…
AssigneeMarvell Asia Pte, Ltd. (Registration No. 199702379M)
InventorsAvinash Sodani, Ulf Hanebutte, Chia-Hsin Chen
Estimating the term…
A method of converting a data stored in a memory from a first format to a second format is disclosed. The method includes extending a number of bits in the data stored in a double data rate (DDR) memory by one bit to form an extended data. The method further includes determining whether the data stored in the DDR is signed or unsigned data. Moreover, responsive to determining that the data is signed, a sign value is added to the most significant bit of the extended data and the data is copied to lower order bits of the extended data. Responsive to determining that the data is unsigned, the data is copied to lower order bits of the extended data and the most significant bit is set to an unsigned value, e.g., zero. The extended data is stored in an on-chip memory (OCM) of a processing tile of a machine learning computer array.
A method of converting a data stored in a memory from a first format to a second format for machine learning (ML) operations, the method comprising:
The method of claim 15, wherein a first scaling value is used for converting the floating point data to an int8 format and wherein a second scaling value is used for converting the floating point data to a uint8 format.
A system comprising:
Dependencies are inferred from the claim text.