US 12,118,056 B2Granted

Methods and apparatus for performing matrix transformations within a memory array

AssigneeMicron Technology, Inc.

InventorFa-Long Luo

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods and apparatus for performing matrix transforms within a memory fabric. Various embodiments of the present disclosure are directed to converting a memory array into a matrix fabric for matrix transformations and performing matrix operations therein. Exemplary embodiments described herein perform matrix transformations within a memory device that includes a matrix fabric and matrix multiplication unit (MMU). In one exemplary embodiment, the matrix fabric uses a “crossbar” construction of resistive elements. Each resistive element stores a level of impedance that represents the corresponding matrix coefficient value. The crossbar connectivity can be driven with an electrical signal representing the input vector as an analog voltage. The resulting signals can be converted from analog voltages to a digital values by an MMU to yield a vector-matrix product. In some cases, the MMU may additionally perform various other logical operations within the digital domain.

Claims23

Dependent claims

An apparatus comprising:

a memory controller;
a plurality of arrays of memory cells, where each memory cell of the plurality of arrays of memory cells are configured to store a digital value as an analog value in an analog medium, wherein:
the plurality of arrays of memory cells comprises a first bank of arrays of the plurality of arrays of memory cells and a second bank of arrays of the plurality of arrays of memory cells,
the first bank of arrays is configured as a memory,
the second bank of arrays is configured as a matrix fabric, and
the plurality of arrays of memory cells are dynamically configurable to be partitioned in either of the first bank of arrays or the second bank of arrays;
and
a memory sense component, where the memory sense component is configured to read the analog value of a first memory cell as a first digital value,
wherein the memory controller is configured to:
receive image data comprising one or more captured color values;
receive a matrix transformation opcode;
operate the second bank of the plurality of arrays of memory cells as a matrix multiplication unit (MMU) based on the matrix transformation opcode, wherein:
each memory cell of the MMU modifies the analog value in the analog medium in accordance with the matrix transformation opcode and a matrix transformation operand,
first read values from a first subset of the memory cells from the first bank are used in a first multiply-accumulate operation to generate a first intermediary vector-matrix product,
the first intermediary vector-matrix product is written back to one of the first subset of the memory cells,
second read values from a second subset of the memory cells from the first bank are used in a second multiply-accumulate operation to generate a second intermediary vector-matrix product,
the second subset is different from the first subset but includes the one of the first subset of the memory cells to which the first intermediary vector-matrix product is written back to, and
the matrix transformation operand comprises the one or more captured color values;
configure the memory sense component to convert the analog value of the first memory cell into a second digital value in accordance with the matrix transformation opcode and the matrix transformation operand;
and
based at least on a reading of the matrix transformation operand into the MMU, write a matrix transformation result based on the second digital value, the matrix transformation result comprising one or more shifted color values respectively associated with the one or more captured color values.

18 more elements

A computerized memory apparatus comprising:

a plurality of memory elements each configured to store digital values as analog values, wherein the plurality of memory elements are dynamically configurable to be partitioned in either a first bank of memory elements or a second bank of memory elements;
a memory sense component configured to read the analog values;
a data interface configured to perform data communication with a processor apparatus;
and
control logic configured to, when operated:
receive one or more instructions for an input transformation from the processor apparatus;
cause configuration of the first bank of memory elements as a matrix multiplication unit (MMU) based at least on the received one or more instructions, wherein the configuration of the first bank of memory elements as the MMU enables atomic matrix fabric computations independent of matrix dimensions;
cause the memory sense component to convert a first analog value, associated with a memory element of the second bank of memory elements configured as a memory, into a first digital value based at least on the received one or more instructions;
and
based at least on the converted first digital value, obtain an output result from the memory sense component;
and
wherein at least the conversion of the first analog value by the memory sense component and the obtainment of the output result each occur while the processor apparatus is in a sleep or low power mode.

9 more elements

A method to perform matrix transformation operations, comprising:

receiving, by a computerized image processing device, video data comprising one or more image blocks, wherein the video data is captured by a camera of the computerized image processing device;
receiving, by the computerized image processing device, a matrix transformation opcode;
configuring, by the computerized image processing device, an array of memory cells of a memory into a matrix structure, based on the matrix transformation opcode, wherein:
a first plurality of the memory cells are configured into the matrix structure,
a second plurality of the memory cells are configured as a memory, and
the first and second plurality of memory cells are dynamically configurable to be partitioned in either of a first subset or a second subset;
reading, by the computerized image processing device, values from a first subset of the second plurality of memory cells for use in a first multiply-accumulate operation to generate a first intermediary vector-matrix product;
writing, by the computerized image processing device, the first intermediary vector-matrix product back to one of the first subset of the first plurality of memory cells;
reading, by the computerized image processing device, values from a second subset of the second plurality of memory cells for use in a second multiply-accumulate operation to generate a second intermediary vector-matrix product, wherein the second subset is different from the first subset but includes the one of the first subset of the first plurality of memory cells to which the first intermediary vector-matrix product is written back to;
configuring, by the computerized image processing device, a memory sense component based on the matrix transformation opcode;
and
based on a reading of a matrix transformation operand into the matrix structure, writing, by the computerized image processing device, a matrix transformation result from the memory sense component, wherein:
(i) the matrix transformation operand comprises the one or more image blocks and is configured to modify one or more analog values of the matrix structure, (ii) the matrix transformation result comprises one or more frequency domain image coefficients, and (iii) the one or more analog values of the matrix structure accumulate the one or more frequency domain image coefficients from the video data over time.

10 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/403,245Filed 3 May 2019
Published: US 2020/0349217 A15 Nov 2020
Granted: 15 Oct 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification11 codes

G06F 17/16
Complex mathematical operations: matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
G06F 7/5443
For evaluating functions by calculation: sum of products
G06F 9/30007
Arrangements for executing specific machine instructions: to perform operations on data operands
G06F 9/30032
To perform operations on data operands: movement instructions, e.g. move, shift, rotate, shuffle

Family

US 2020/0349217 A1Pre-grant publication · published 5 Nov 2020
US 12,118,056 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key