# US 11,907,065 B2 · Resiliency and performance for cluster memory > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,065 B2 - Document ID: US11907065B2 - Status: Granted - Kind: B2 - Application number: 18/101,536 - Filed: 2023-01-25 - Published: 2023-06-01 - Granted: 2024-02-20 - Earliest claimed: 2021-07-16 - Assignees: VMware, Inc. - Applicants: VMware LLC - Inventors: Marcos K. Aguilera; Keerthi Kumar; Pramod Kumar; Pratap Subrahmanyam; Sairam Veeraswamy; Rajesh Venkatasubramanian - Web page: https://patentgrep.com/patent/US11907065B2 ## CPC - G06F 11/1068: {in sector programmable memories, e.g. flash disk (G06F11/1072 takes precedence)} - G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors} - G06F 3/064: {Management of blocks} - G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 3/0673: {Single storage device} - G06F 3/0689: {Disk arrays, e.g. RAID, JBOD} - G06F 11/0772: {Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers} - G06F 11/108: {Parity data distribution in semiconductor storages, e.g. in SSD} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed are various embodiments for improving the resiliency and performance of clustered memory. A computing device can generate at least one parity page from at least a first local page and a second local page. The computing device can then submit a first write request for the first local page to a first one of a plurality of memory hosts. The computing device can also submit a second write request for the second local page to a second one of the plurality of memory hosts. Additionally, the computing device can submit a third write request for the parity page to a third one of the plurality of memory hosts. ## Claims ### Claim 1 A system, comprising: a computing device comprising a processor and a memory; and machine-readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least: generate at least one parity subpage for a respective one of a plurality of subpages of a local page in response to a change to the local page; submit a write request for each of the plurality of subpages to a respective one of a plurality of memory hosts; and submit an additional write request for the at least one parity subpage to an additional one of the plurality of memory hosts. ### Claim 2 (depends on 1) The system of claim 1, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least: determine that one of the plurality of memory hosts storing at least one of the plurality of subpages or the at least one parity subpage has faulted; retrieve a minimum number of remaining subpages from the combination of the plurality of subpages and the at least one parity page; and reconstruct a missing subpage or parity subpage from the minimum number of remaining subpages. ### Claim 3 (depends on 2) The system of claim 2, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least: select a new memory host to store the missing subpage or parity subpage; and submit a third write request to the new memory host for the missing subpage or parity subpage. ### Claim 4 (depends on 1) The system of claim 1, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least: in response to a page-fault, retrieve a minimum number of subpages from the combination of the plurality of subpages and the at least one parity page; reconstruct the local page based at least in part on the minimum number of subpages; and load the local page into the memory of the computing device. ### Claim 5 (depends on 1) The system of claim 1, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least record in a local chunk map the chunks in which the plurality of subpages and the at least one parity page are stored and the offsets within the chunks in which the plurality of subpages and the at least one parity subpage are stored. ### Claim 6 (depends on 1) The system of claim 1, wherein the additional write request for the at least one parity subpage to the additional one of the plurality of memory hosts specifies that the at least one parity subpage is to be stored on a disk of the additional one of the plurality of memory hosts. ### Claim 7 (depends on 1) The method implemented with the computer of claim 1, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least communicate a heartbeat to the plurality of memory hosts. ### Claim 8 A method, comprising: generating at least one parity subpage for a respective one of a plurality of subpages of a local page in response to a change to the local page; submitting a write request for each of the plurality of subpages to a respective one of a plurality of memory hosts; and submitting an additional write request for the at least one parity subpage to an additional one of the plurality of memory hosts. ### Claim 9 (depends on 8) The method of claim 8, further comprising: determining that one of the plurality of memory hosts storing at least one of the plurality of subpages or the at least one parity subpage has faulted; retrieving a minimum number of remaining subpages from the combination of the plurality of subpages and the at least one parity page; and reconstructing a missing subpage or parity subpage from the minimum number of remaining subpages. ### Claim 10 (depends on 9) The method of claim 9, further comprising: selecting a new memory host to store the missing subpage or parity subpage; and submitting a third write request to the new memory host for the missing subpage or parity subpage. ### Claim 11 (depends on 8) The method of claim 8, further comprising: in response to a page-fault, retrieving a minimum number of subpages from the combination of the plurality of subpages and the at least one parity page; reconstructing the local page based at least in part on the minimum number of subpages; and loading the local page into the memory of the computing device. ### Claim 12 (depends on 8) The method of claim 8, further comprising recording in a local chunk map the chunks in which the plurality of subpages and the at least one parity page are stored and the offsets within the chunks in which the plurality of subpages and the at least one parity subpage are stored. ### Claim 13 (depends on 8) The method of claim 8, wherein the additional write request for the at least one parity subpage to the additional one of the plurality of memory hosts specifies that the at least one parity subpage is to be stored on a disk of the additional one of the plurality of memory hosts. ### Claim 14 (depends on 8) The method of claim 8, further comprising communicating a heartbeat to the plurality of memory hosts. ### Claim 15 A non-transitory, computer-readable medium comprising machine-readable instructions that, when executed by a processor of a computing device, cause the computing device to at least: generate at least one parity subpage for a respective one of a plurality of subpages of a local page in response to a change to the local page; submit a write request for each of the plurality of subpages to a respective one of a plurality of memory hosts; and submit an additional write request for the at least one parity subpage to an additional one of the plurality of memory hosts. ### Claim 16 (depends on 15) The non-transitory, computer-readable medium of claim 15, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least: determine that one of the plurality of memory hosts storing at least one of the plurality of subpages or the at least one parity subpage has faulted; retrieve a minimum number of remaining subpages from the combination of the plurality of subpages and the at least one parity page; and reconstruct a missing subpage or parity subpage from the minimum number of remaining subpages. ### Claim 17 (depends on 16) The non-transitory, computer-readable medium of claim 16, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least: select a new memory host to store the missing subpage or parity subpage; and submit a third write request to the new memory host for the missing subpage or parity subpage. ### Claim 18 (depends on 15) The non-transitory, computer-readable medium of claim 15, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least: in response to a page-fault, retrieve a minimum number of subpages from the combination of the plurality of subpages and the at least one parity page; reconstruct the local page based at least in part on the minimum number of subpages; and load the local page into the memory of the computing device. ### Claim 19 (depends on 15) The non-transitory, computer-readable medium of claim 15, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least record in a local chunk map the chunks in which the plurality of subpages and the at least one parity page are stored and the offsets within the chunks in which the plurality of subpages and the at least one parity subpage are stored. ### Claim 20 (depends on 15) The non-transitory, computer-readable medium of claim 15, wherein the machine-readable instructions, when executed by the processor, further cause the computing device to at least communicate a heartbeat to the plurality of memory hosts.