# US 2024/0184526 A1 · Memory device and operating method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0184526 A1
- Document ID: US20240184526A1
- Status: Published application
- Kind: A1
- Application number: 18/329,045
- Filed: 2023-06-05
- Published: 2024-06-06
- Earliest claimed: 2022-12-02
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Seungwoo SEO; Sanghoon CHA
- Web page: https://patentgrep.com/patent/US20240184526A1

## CPC

- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 15/7821: {Tightly coupled to memory, e.g. computational memory, smart memory, processor in memory}
- G06F 7/502: Half adders; Full adders consisting of two cascaded half adders {(G06F7/5013 takes precedence)}
- G06F 9/30145: {Instruction analysis, e.g. decoding, instruction word fields}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A memory device includes: a plurality of memory banks divided by a plurality of channels comprising a first channel and a second channel; and a channel-level processing element (PE) configured to generate an in-memory computation result by performing an operation using a first partial result generated based on data stored in a memory bank of the first channel among the plurality of memory banks and a second partial result generated based on data stored in a memory bank of the second channel among the plurality of memory banks.

## Claims

### Claim 1

A memory device comprising:
a plurality of memory banks divided by a plurality of channels comprising a first channel and a second channel; and
a channel-level processing element (PE) configured to generate an in-memory computation result by performing an operation using a first partial result generated based on data stored in a memory bank of the first channel among the plurality of memory banks and a second partial result generated based on data stored in a memory bank of the second channel among the plurality of memory banks.

### Claim 2 (depends on 1)

The memory device of claim 1, wherein the channel-level PE is configured to transmit the in-memory computation result to a host processor.

### Claim 3 (depends on 1)

The memory device of claim 1, wherein the channel-level PE comprises:
a plurality of operators configured to apply an operation according to an instruction to partial operation results of the plurality of channels respectively; and
an adder configured to, for the generating of the in-memory computation result, determine a sum of outputs of the plurality of operators of the plurality of channels.

### Claim 4 (depends on 3)

The memory device of claim 3, wherein the channel-level PE comprises:
a register configured to register the outputs of the plurality of operators;
an instruction memory configured to store the instruction; and
a decoder configured to control the plurality of operators, the adder, and the register by interpreting the instruction.

### Claim 5 (depends on 3)

The memory device of claim 3, wherein, for the generating of the in-memory computation result, channel-level PE is configured to:
obtain a partial result corresponding to an operation on values stored in a respective memory channel among a plurality of operations belonging to the same task, during parallel execution of a plurality of tasks;
store a partial result corresponding to a respective memory channel in a register area allocated for each of the tasks in a register corresponding to the respective memory channel; and
generate the in-memory computation result by loading partial results generated for the same task in a plurality of memory channels from the register corresponding to the channel, and determining a sum of the loaded partial results through the adder.

### Claim 6 (depends on 3)

The memory device of claim 3, wherein, for the generating of the in-memory computation result, channel-level PE is configured to:
based on a sum of products of values of memory banks belonging to the same memory channel, receive an intermediate result corresponding to the memory channel;
accumulate received intermediate results by the plurality of operators;
store a partial result obtained by the accumulating in a register; and
generate the in-memory computation result by determining a sum of partial results stored in the register through the adder.

### Claim 7 (depends on 3)

The memory device of claim 3, wherein
an intra-channel PE of the memory device is configured to generate a first sum result by determining a sum of embedding vectors of banks in the same rank among embedding vectors selected as the same segment in a tensor to be determined, and
for the generating of the in-memory computation result, the channel-level PE is configured to:
obtain an intermediate result based on a sum of the first sum result and another first sum result in the same channel among the embedding vectors selected as the same segment;
generate a partial result by accumulating intermediate results obtained from respective channels, and store a generated partial result in a register; and
generate the in-memory computation result by determining a sum of partial results.

### Claim 8 (depends on 1)

The memory device of claim 1, wherein, for the performing of the operation, the channel-level PE is configured to perform any one or any combination of any two or more operations among addition, subtraction, multiplication, division, multiply-accumulate (MAC) operation, and reduction operation on inter-channel data.

### Claim 9 (depends on 1)

The memory device of claim 1, further comprising an intra-channel PE configured to generate a partial result by performing an operation on data of memory banks belonging to the same channel among the plurality of memory banks.

### Claim 10 (depends on 9)

The memory device of claim 9, wherein, for the performing of operation on data of the memory banks, the intra-channel PE is configured to perform any one or any combination of any two or more operations among addition, subtraction, multiplication, division, MAC operation, and reduction operation on intra-channel data.

### Claim 11 (depends on 1)

The memory device of claim 1, further comprising a bank-level PE configured to perform an operation on data of memory banks belonging to the same rank among the plurality of memory banks.

### Claim 12 (depends on 1)

The memory device of claim 1, further comprising a rank-level PE configured to perform an operation between different memory ranks belonging to the same channel among a plurality of memory ranks.

### Claim 13 (depends on 1)

The memory device of claim 1, further comprising:
a first bank-level PE configured to generate a first bank-level result through an operation on data stored in different banks in the first channel; and
a first rank-level PE configured to generate the first partial result based on the first bank-level result generated by the first bank-level PE.

### Claim 14 (depends on 1)

The memory device of claim 1, further comprising:
a second bank-level PE configured to generate a second bank-level result through an operation on data stored in different banks in the second channel; and
a second rank-level PE configured to generate the second partial result based on the second bank-level result generated by the second bank-level PE.

### Claim 15 (depends on 1)

The memory device of claim 1, wherein the plurality of memory banks forms a tree structure.

### Claim 16 (depends on 1)

The memory device of claim 1, wherein
the channel-level PE is in a logic die of the memory device, and
the plurality of memory banks is in a core die of the memory device.

### Claim 17 (depends on 16)

The memory device of claim 16, wherein
the logic die and a plurality of memory core dies are stacked, and
partial results of memory channels in the plurality of memory core dies are collectively processed in the channel-level PE in the logic die.

### Claim 18 (depends on 16)

The memory device of claim 16, wherein a computation result obtained by the channel-level PE in the logic die is transmitted to a host processor of a processor die through an interposer.

### Claim 19 (depends on 1)

The memory device of claim 1, wherein
the channel-level PE comprises a plurality of operators each corresponding to a respective one of the channels and configured to perform an operation between a partial result corresponding to the channel and a previously generated partial result stored in a register corresponding to the operator, and
an adder configured to, for the generating of the in-memory computation result, determine a sum of outputs of the plurality of operators.

### Claim 20 (depends on 2)

An electronic device comprising:
the memory device of claim 2; and
the host processor.

### Claim 21

A processor-implemented method of operating a memory device, comprising:
generating a first partial result based on data stored in a memory bank of a first channel among a plurality of memory banks and generating a second partial result based on data stored in a memory bank of a second channel among the plurality of memory banks; and
generating an in-memory computation result by performing an operation using the first partial result and the second partial result by a channel-level processing element (PE).

### Claim 22 (depends on 21)

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 21.

### Claim 23

A memory device comprising:
a first memory rank comprising a bank-level processing element (PE) configured to generate a bank-level result through an operation on data stored in banks of the first memory rank;
a first memory channel comprising the first memory rank, a second memory rank, and an intra-channel PE configured to generate a partial result by performing an operation on the bank-level result of the first memory rank and a bank-level result of the second memory rank; and
a channel-level PE configured to generate an in-memory computation result by performing an operation using the partial result of the first memory channel and a partial result of a second memory channel, and to transmit the in-memory computation result to a host processor.
