Techniques to support transformer models in analog compute-in-memory hardware
OwnerJames Read, Shimeng Yuinventors
Life of this patent
Estimating the term…
Loading patent…
OwnerJames Read, Shimeng Yuinventors
Estimating the term…
The present disclosure provides a method for implementing transformer models in analog compute-in-memory hardware. The method comprises training a target neural network using one or more operators on one or more graphics processing units, generating one or more datasets from full network traces to capture input-output relationships of non-vector-matrix multiplication operations, training one or more multi-layer perceptrons to approximate the non-vector-matrix multiplication operations using the one or more datasets, replacing the original non-vector-matrix multiplication operations with the trained one or more multi-layer perceptrons, and mapping the resulting multi-layer perceptron-only neural network to an analog compute-in-memory architecture. The non-vector-matrix multiplication operations comprise layer normalization operations, softmax operations, and GELU activation operations. The analog compute-in-memory architecture comprises crossbar arrays of memory elements that store weight values as analog quantities using conductance or capacitance properties.
A method for implementing transformer models in analog compute-in-memory hardware, comprising:
A neural network system comprising:
A neural network system comprising:
A neural network system comprising:
Dependencies are inferred from the claim text.