Systems and methods for attention masking
AssigneeTesla, Inc.
InventorsRitvik RAWAT, Srihari SAMPATHKUMAR
Life of this patent
Estimating the term…
Abstract
Embodiments described herein provide for computing masked scores in transformer models without materializing a full mask. A matrix is partitioned into submatrices based on available processing elements. At least one element is initialized with row and column index values and evaluates a masking condition using a comparison operation, which is fused into the softmax computation. This approach supports causal and batched masks with minimal overhead and is well-suited for applications such as autonomous navigation, where computed weights guide downstream decision-making.
Claims20
A method for autonomous navigation of an ego using a transformer model, the method comprising:
- partitioning, by one or more processors, a matrix associated with the transformer model into a plurality of submatrices based on a number of processing elements;
- assigning, by the one or more processors, the plurality of submatrices to a plurality of processing elements configured to generate a score for the plurality of submatrices;
- initializing, by the one or more processors, a column index value and a row index value for the plurality of submatrices corresponding to a respective position in the matrix;
- generating, by the one or more processors, a score in accordance with a softmax computation and a masking condition based on the row index value and the column index value;
- and
- transmitting, by the one or more processors, to an autonomous navigation application, at least one weight based on the score, whereby the autonomous navigation application generate at least one executable navigational instruction to be executed by a processor of the ego.
A computer system for autonomous navigation of an ego using a transformer model, the computer system comprising a computer-readable medium having a set of non-transitory instructions that when executed by at least one processor, cause the at least one processor to:
- partition a matrix associated with the transformer model into a plurality of submatrices based on a number of processing elements;
- assign the plurality of submatrices to a plurality of processing elements configured to generate a score for the plurality of submatrices;
- initialize a column index value and a row index value for the plurality of submatrices corresponding to a respective position in the matrix;
- generate a score in accordance with a softmax computation and a masking condition based on the row index value and the column index value;
- and
- transmit to an autonomous navigation application, at least one weight based on the score, whereby the autonomous navigation application generate at least one executable navigational instruction to be executed by a processor of the ego.
A computer system for autonomous navigation of an ego using a transformer model, the computer system comprising at least one processor configured to:
- partition a matrix associated with the transformer model into a plurality of submatrices based on a number of processing elements;
- assign the plurality of submatrices to a plurality of processing elements configured to generate a score for the plurality of submatrices;
- initialize a column index value and a row index value for the plurality of submatrices corresponding to a respective position in the matrix;
- generate a score in accordance with a softmax computation and a masking condition based on the row index value and the column index value;
- and
- transmit to an autonomous navigation application, at least one weight based on the score, whereby the autonomous navigation application generate at least one executable navigational instruction to be executed by at least one processor of the ego.
Dependencies are inferred from the claim text.