# US 11,908,546 B2 · In-memory lightweight memory coherence protocol > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,546 B2 - Document ID: US11908546B2 - Status: Granted - Kind: B2 - Application number: 17/066,422 - Filed: 2020-10-08 - Published: 2021-04-29 - Granted: 2024-02-20 - Earliest claimed: 2014-05-08 - Assignees: Micron Technology, Inc. - Applicants: Micron Technology, Inc. - Inventors: Richard C Murphy - Web page: https://patentgrep.com/patent/US11908546B2 ## CPC - G11C 8/12: Group selection circuits, e.g. for memory block selection, chip selection, array selection - G06F 12/0284: {Multiple user address space allocation, e.g. using different base addresses (interprocessor communication G06F15/163)} - G06F 12/0815: Cache consistency protocols - G06F 12/0817: using directory methods - G06F 12/1072: Decentralised address translation, e.g. in distributed shared memory systems - G11C 5/02: Disposition of storage elements, e.g. in the form of a matrix array - G11C 5/025: {Geometric lay-out considerations of storage- and peripheral-blocks in a semiconductor storage device (geometrical lay-out of the components in integrated circuits, geometrical lay-out of the components in integrated circuits H10D89/10)} - G06F 12/0824: {Distributed directories, e.g. linked lists of caches} - G06F 12/084: with a shared cache - G06F 2212/1032: Reliability improvement, data loss prevention, degraded operation etc - G06F 2212/1048: Scalability - G06F 2212/251: Local memory within processor subsystem - G06F 2212/3042: being part of a memory device, e.g. cache DRAM - G11C: STATIC STORES (semiconductor memory devices H10B) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system includes a plurality of host processors and a plurality of HMC devices configured as a distributed shared memory for the host processors. An HMC device includes a plurality of integrated circuit memory die including at least a first memory die arranged on top of a second memory die and at least a portion of the memory of the memory die is mapped to include at least a portion of a memory coherence directory; and a logic base die including at least one memory controller configured to manage three-dimensional (3D) access to memory of the plurality of memory die by at least one second device, and logic circuitry configured to determine memory coherence state information for data stored in the memory of the plurality of memory die, communicate information regarding the access to memory, and include the memory coherence information in the communicated information. ## Claims ### Claim 1 A computing system comprising: multiple memory devices configured as a system distributed shared memory for at least one host processing device of the computing system, wherein input/output (I/O) information is communicated among the multiple memory devices and the at least one host processing device using a packetized I/O protocol, and wherein each memory device includes: multiple memory dies, wherein at least a portion of the memory of the memory dies is mapped to include at least a portion of a memory coherence directory; and a logic base die excluding the at least one host processing device and including at least one memory controller configured to manage access to memory of the memory dies by the at least one host processing device, and logic circuitry configured to: determine memory coherence state information for data stored in the memory of one or more of the memory dies; store the memory coherence state information with the data in the memory dies; grant access to the stored memory coherence state information in response to an atomic memory request, wherein the stored memory coherence information for a memory word is accessed by the logic circuitry remapping an address of the memory word received in the atomic memory request; and include the memory coherence state information in a packetized message of the packetized I/O protocol sent in response to the atomic memory request. ### Claim 2 (depends on 1) The computing system of claim 1, including at least one memory device configured as a hub memory device that includes: at least one pass-through link configured to route packetized messages; and logic circuitry configured to snoop memory coherence state information in the routed packetized messages. ### Claim 3 (depends on 1) The computing system of claim 1, wherein the logic circuitry of each memory device is configured to: broadcast a packetized message that includes the memory coherence state information using the packetized I/O protocol; and snoop the memory coherence state information included in packetized messages broadcast by other memory devices. ### Claim 4 (depends on 1) The computing system of claim 1, wherein the logic circuitry of each memory device is configured to: route packetized information as part of the packetized I/O protocol; and detect memory coherence state information included in the routed packetized information. ### Claim 5 (depends on 1) The computing system of claim 1, wherein the logic circuitry of each memory device is configured to: determine modified state information, shared state information, and invalid state information for a word of memory of the memory dies; and store the state information in association with the word of memory of the memory dies. ### Claim 6 (depends on 1) The computing system of claim 1, wherein the logic circuitry of each memory device is configured to: determine at least one of exclusivity state information and ownership state information for a word of memory of the memory die; and store the state information in association with the word of shared memory data. ### Claim 7 (depends on 1) The computing system of claim 1, including: at least one memory device configured as a hub memory device; and wherein logic circuitry of the hub memory device adds routing information to a packetized message and monitors memory coherence state information in other packetized messages. ### Claim 8 (depends on 1) The computing system of claim 1, wherein the memory controller of the logic base die of each memory is configured to manage access to memory of the memory dies by the at least one host processing device and by another memory device. ### Claim 9 A method of operating a computer system that includes system processing devices and a distributed shared memory that excludes the system processing devices, the method comprising: determining, by the distributed shared memory, memory coherence state information resulting from memory access by the system processing devices and storing the memory coherence state information with the data in the distributed shared memory; communicating packetized information among the system processing devices and multiple shared memory endpoints of the computing system using a packetized input/output (I/O) protocol; initiating, by the distributed shared memory according to the packetized I/O protocol, sending of a packetized message including the stored memory coherence state information in response to an atomic request to access the memory coherence state information stored in the distributed shared memory, wherein the stored memory coherence state information for a memory word is accessed by the distributed shared memory remapping an address of the memory word received in the atomic memory request; and tracking the memory coherence state information using the shared memory endpoints. ### Claim 10 (depends on 9) The method of claim 9, wherein tracking the memory coherence state information using the shared memory endpoints includes snooping the memory coherence state information in the packetized message using memory devices that include the shared memory. ### Claim 11 (depends on 9) The method of claim 9, including storing the memory coherence state information with a shared memory data word. ### Claim 12 (depends on 9) The method of claim 9, including modifying the memory coherence state information using a logic base layer that includes at least one memory controller for the shared memory. ### Claim 13 (depends on 9) The method of claim 9, wherein determining memory coherence state information by the shared memory includes storing a modified bit, a shared bit, and an invalid bit with the shared memory data word. ### Claim 14 (depends on 9) The method of claim 9, including: receiving, from a separate device by a shared memory endpoint, a non-modifying request to a memory word; and sending, by the shared memory endpoint, a packetized response communication to the non-modifying request, wherein the packetized response includes memory state information of the memory word. ### Claim 15 (depends on 9) The method of claim 9, including: receiving, from a separate device by a shared memory endpoint, a request for memory access to a memory word; setting, by the shared memory endpoint, memory state information of the memory word according to the request; sending, by the shared memory endpoint, a packetized response communication to the request, wherein the packetized response includes the memory state information of the memory word. ### Claim 16 An electronic device of a single electronic device package, the electronic device including: multiple integrated circuit memory dies configured as a system shared memory, and at least a portion of the system shared memory of the memory dies mapped to include at least a portion of a memory coherence directory; and wherein the system shared memory includes a logic base die included in the single electronic device package and separate from the logic base die excluding a system processor, the logic base die including at least one memory controller configured to manage access to the system shared memory by at least one separate device, and logic circuitry configured to determine memory coherence state information for data stored in the system shared memory, grant access to stored memory coherence state information stored in the system shared memory in response to an atomic memory request, wherein the stored memory coherence state information for a memory word is accessed by the logic circuitry remapping an address of the memory word included in the atomic memory request, and include the stored memory coherence state information in a packetized message of a packetized input/output (I/O) protocol that is sent in response to the atomic memory request. ### Claim 17 (depends on 16) The electronic device of claim 16, wherein the logic circuitry of the logic base die is configured to broadcast the memory coherence state information using the packetized I/O protocol. ### Claim 18 (depends on 17) The electronic device of claim 17, wherein the logic circuitry of the logic base die is configured to: route packetized information as part of the I/O protocol; snoop memory coherence state information included with the routed packetized information; and update memory coherence state information stored by the electronic device according to the monitored state information. ### Claim 19 (depends on 16) The electronic device of claim 16, wherein the logic circuitry of the logic base die is configured to determine modified state information, shared state information, and invalid state information for a word of memory of the memory die and store the state information in association with the word of memory of the memory die. ### Claim 20 (depends on 16) The electronic device of claim 16, wherein the logic circuitry of the logic base die is configured to determine at least one of exclusivity state information and ownership state information for a word of memory of the memory die and store the state information in association with the word of shared memory data.