Back to results
US 12,664,414 B2Granted
Compute in memory-based machine learning accelerator architecture
AssigneeQUALCOMM Incorporated
InventorRen Li
Life of this patent
Estimating the term…
Abstract
Certain aspects of the present disclosure provide techniques for processing machine learning model data with a machine learning task accelerator, including: configuring one or more signal processing units (SPUs) of the machine learning task accelerator to process a machine learning model; providing model input data to the one or more configured SPUs; processing the model input data with the machine learning model using the one or more configured SPUs; and receiving output data from the one or more configured SPUs.
Claims30
A machine learning task accelerator, comprising:
- one or more mixed signal processing units (MSPUs), each respective MSPU of the one or more MSPUs comprising:
- a compute-in-memory (CIM) circuit;
- a local activation buffer connected to the CIM circuit and configured to store activation data for processing by the CIM circuit;
- one or more analog to digital converters (ADCs) connected to the CIM circuit and configured to convert analog computation result signals from the CIM circuit to digital computation result data;
- a first nonlinear operation circuit connected to one or more outputs of the one or more ADCs and configured to perform nonlinear processing on the digital computation result data;
- a hardware sequencer circuit configured to execute instructions received from a host system and control operation of the MSPU;
- and
- a local direct memory access (DMA) controller configured to control access to a shared activation buffer;
- a digital multiplication and accumulation (DMAC) circuit connected to the one or more MSPUs and configured to perform multiplication and accumulation operations on activation data output from one or more of the one or more MSPUs;
- a digital element-wise multiplication and accumulation circuit connected to the one or more MSPUs and configured to:
- perform element-wise multiplication and element-wise accumulation operations on activation data output from one or more of the one or more MSPUs;
- and
- generate an output signal based on the element-wise multiplication and element-wise accumulation operations, wherein the output signal is provided back as input to the digital element-wise multiplication and accumulation circuit via a delay loop;
- and
- a second nonlinear operation circuit connected to the one or more MSPUs, wherein the second nonlinear operation circuit is configured to receive the output signal.
12 more elements
A method of processing machine learning model data with a machine learning task accelerator, comprising:
- configuring one or more mixed signal processing units (MSPUs) of the machine learning task accelerator to process a machine learning model;
- providing model input data to the one or more configured MSPUs;
- processing the model input data with the machine learning model using the one or more configured MSPUs;
- and
- receiving output data from the one or more configured MSPUs, wherein the machine learning task accelerator comprises:
- a digital multiplication and accumulation (DMAC) circuit connected to the one or more MSPUs and configured to perform multiplication and accumulation operations on activation data output from one or more of the one or more MSPUs;
- a digital element-wise multiplication and accumulation circuit connected to the one or more MSPUs and configured to:
- perform element-wise multiplication and element-wise accumulation operations on activation data output from one or more of the one or more MSPUs;
- and
- generate an output signal based on the element-wise multiplication and element-wise accumulation operations, wherein the output signal is provided back as input to the digital element-wise multiplication and accumulation circuit via a delay loop;
- and
- a second nonlinear operation circuit connected to the one or more MSPUs, wherein the second nonlinear operation circuit is configured to receive the output signal.
9 more elements
Dependencies are inferred from the claim text.