# US 2021/0342734 A1 · System and method for int9 quantization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342734 A1
- Document ID: US20210342734A1
- Status: Published application
- Kind: A1
- Application number: 16/862,549
- Filed: 2020-04-29
- Published: 2021-11-04
- Earliest claimed: 2020-04-29
- Assignees: Marvell Asia Pte, Ltd. (Registration No. 199702379M)
- Applicants: Marvell Asia Pte, Ltd. (Registration No. 199702379M)
- Inventors: Avinash Sodani; Ulf Hanebutte; Chia-Hsin Chen
- Web page: https://patentgrep.com/patent/US20210342734A1

## CPC

- G06N 3/063: using electronic means
- G06N 20/00: Machine learning
- G06F 9/30014: {with variable precision}
- G06F 9/30025: {Format conversion instructions, e.g. Floating-Point to Integer, decimal conversion}
- G06N 3/048: Activation functions
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method of converting a data stored in a memory from a first format to a second format is disclosed. The method includes extending a number of bits in the data stored in a double data rate (DDR) memory by one bit to form an extended data. The method further includes determining whether the data stored in the DDR is signed or unsigned data. Moreover, responsive to determining that the data is signed, a sign value is added to the most significant bit of the extended data and the data is copied to lower order bits of the extended data. Responsive to determining that the data is unsigned, the data is copied to lower order bits of the extended data and the most significant bit is set to an unsigned value, e.g., zero. The extended data is stored in an on-chip memory (OCM) of a processing tile of a machine learning computer array.

## Claims

### Claim 1

A method of converting a data stored in a memory from a first format to a second format for machine learning (ML) operations, the method comprising:
extending a number of bits in the data stored in a double data rate (DDR) memory by one bit to form an extended data;
determining whether the data stored in the DDR memory is signed or unsigned data;
responsive to determining that the data is signed, adding a sign value to the most significant bit of the extended data and copying the data to lower order bits of the extended data;
responsive to determining that the data is unsigned, copying the data to lower order bits of the extended data and setting the most significant bit to an unsigned value; and
storing the extended data in an on-chip memory (OCM) of a processing tile of a machine learning computer array.

### Claim 2 (depends on 1)

The method of claim 1, wherein the data is an unsigned integer.

### Claim 3 (depends on 1)

The method of claim 1, wherein the data is a signed integer.

### Claim 4 (depends on 1)

The method of claim 1, wherein the data is 8 bits and wherein the extended data is 9 bits.

### Claim 5 (depends on 1)

The method of claim 1, wherein the extended data is an int9 data.

### Claim 6 (depends on 1)

The method of claim 1 further comprising:
tracking whether the data stored in the DDR memory is signed or unsigned; and
scheduling appropriate instructions for the extended data based on whether the data is signed or unsigned.

### Claim 7 (depends on 6)

The method of claim 6 further comprising performing an arithmetic logic unit (ALU) operation on the extended data as an operand.

### Claim 8 (depends on 7)

The method of claim 7 further comprising storing a result of the operation in the OCM of the processing tile of the machine learning computer array.

### Claim 9 (depends on 8)

The method of claim 8 further comprising storing the result stored in the OCM into the DDR memory.

### Claim 10 (depends on 9)

The method of claim 9 further comprising: prior to storing the result in the DDR memory, adjusting a value of the result to a maximum value of a range for the data if the value of the result exceeds the maximum value and adjusting the value of the result to a minimum value of the range for the data if the value of the result is lower than a minimum value of the range for the data.

### Claim 11 (depends on 9)

The method of claim 9 further comprising dropping the most significant bit of the result before storing the result in the DDR memory from the OCM.

### Claim 12 (depends on 1)

The method of claim 1, wherein the data stored in the DDR memory is an integer representation of the data for a floating point data.

### Claim 13 (depends on 12)

The method of claim 12, wherein the floating point data is scaled and quantized to form the data in the first format.

### Claim 14 (depends on 15)

The method of claim 15, wherein a first scaling value is used for converting the floating point data to an int8 format and wherein a second scaling value is used for converting the floating point data to a uint8 format.

### Claim 15

A system comprising:
a double data rate (DDR) memory configured to store integer data in a first format; and
a machine learning processing unit comprising a plurality of processing tiles, wherein each processing tile comprises:
an on-chip memory (OCM) configured to accept and maintain an extended data that is converted from the integer data in the first format from the DDR memory, for various ML operations, wherein the extended data includes one additional bit in comparison to the integer data in the first format, and wherein the most significant bit of the extended data is signed if the integer data in the first format is signed and wherein the most significant bit of the extended data is set to an unsigned value if the integer data in the first format is unsigned and wherein the least significant bits of the extended data is the same as the integer data in the first format.

### Claim 16 (depends on 15)

The system of claim 15, wherein the integer data in the first format is either int8 or uint8.

### Claim 17 (depends on 15)

The system of claim 15, wherein the extended data is int9.

### Claim 18 (depends on 15)

The system of claim 15, wherein whether the integer data in the first format stored in the DDR memory is signed or unsigned is tracked and wherein appropriate instructions are scheduled depending whether the integer data in the first format is signed or unsigned.

### Claim 19 (depends on 15)

The system of claim 15, wherein the extended data is an operand for an operation.

### Claim 20 (depends on 19)

The system of claim 19, wherein a result of the operation is stored in the OCM.

### Claim 21 (depends on 20)

The system of claim 20, wherein the result of the operation that is stored in the OCM is further stored in the DDR memory.

### Claim 22 (depends on 21)

The system of claim 21, wherein a value of the result is adjusted to a maximum value of a range for the integer data in the first format if the value of the result exceeds the maximum value and adjusting the value of the result to a minimum value of the range for the integer data in the first format if the value of the result is lower than a minimum value of the range for the data, prior to storing the result in the DDR memory.

### Claim 23 (depends on 21)

The system of claim 21, wherein the most significant bit of the result is dropped before storing the result in the DDR memory.

### Claim 24 (depends on 15)

The system of claim 15, wherein the integer data in the first format is integer representation of a floating point data, and wherein the floating point data is scaled and quantized to form the integer data in the first format.

### Claim 25 (depends on 24)

The system of claim 24, wherein a first scaling value is used for converting the floating point data to an int8 format and wherein a second scaling value is used for converting the floating point data to a uint8 format.
