Systems and methods for attention computation in quantized transformer models

A computer system for navigating an ego via ingesting sensor data, the computer system comprising a computer readable medium having a set of non-transitory instructions, that when executed by at least one processor, cause the at least one processor to:

• receive a set of quantized values resulting from a matrix multiplication of quantized query and key vectors in an attention layer of a transformer model, the set of quantized values corresponding to sensor data associated with the ego;
• identify from the set of quantized values, a maximum value using an integer comparison operation;
• for at least one quantized value in the set of quantized values, compute a difference between the quantized value and the maximum value;
• execute a bit shift operation, on the difference between the quantized value and the maximum value, corresponding to a power-of-two quantization scale to generate bit-shifted values;
• execute a base-2 exponentiation operation to the bit shifted values to generate base-2 exponentiated values;
• aggregate the base-2 exponentiated values to generate an aggregated value;
• and
• execute a model using the aggregated value to output a navigational instruction to be executed by the ego.

A computer system for navigating an ego via ingesting sensor data, the computer system comprising at least one processor configured to:

• receive a set of quantized values resulting from a matrix multiplication of quantized query and key vectors in an attention layer of a transformer model, the set of quantized values corresponding to sensor data associated with the ego;
• identify from the set of quantized values, a maximum value using an integer comparison operation;
• for at least one quantized value in the set of quantized values, compute a difference between the quantized value and the maximum value;
• execute a bit shift operation, on the difference between the quantized value and the maximum value, corresponding to a power-of-two quantization scale to generate bit-shifted values;
• execute a base-2 exponentiation operation to the bit shifted values to generate base-2 exponentiated values;
• aggregate the base-2 exponentiated values to generate an aggregated value;
• and
• execute a model using the aggregated value to output a navigational instruction to be executed by the ego.