# US 12,117,895 B2 · Memory error recovery using write instruction signaling > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,117,895 B2 - Document ID: US12117895B2 - Status: Granted - Kind: B2 - Application number: 17/709,947 - Filed: 2022-03-31 - Published: 2023-10-05 - Granted: 2024-10-15 - Earliest claimed: 2022-03-31 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Jue Wang - Web page: https://patentgrep.com/patent/US12117895B2 ## CPC - G06F 11/0793: {Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40)} - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 11/0709: {in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems} - G06F 11/0712: {in a virtual computing platform, e.g. logically partitioned systems} - G06F 11/073: {in a memory management context, e.g. virtual memory or cache management (memory management G06F12/00; testing of static memory units G11C29/00)} - G06F 11/0772: {Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers} - G06F 11/1484: {involving virtual machines} - G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)} - G06F 11/0784: {Routing of error reports, e.g. with a specific transmission path or data flow} - G06F 2009/4557: {Distribution of virtual machine instances; Migration and load balancing} - G06F 2009/45583: {Memory management, e.g. access or allocation} - G06F 2201/805: Real-time - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method for balancing data storage among a plurality of groups of computing devices, each group comprising one or more respective computing devices, each group having an available storage capacity. The method may involve, for each group of computing devices, determining an amount of used storage at the group of computing devices exceeding a predefined first threshold value that is less than the available storage capacity and calculating a storage cost based on the determined amount of used storage exceeding the predefined first threshold value, determining a total storage cost of the plurality of groups of computing devices based on a sum of the calculated storage costs, determining a transfer of one or more projects between the groups of computing devices that reduces the total storage and directing the plurality of groups of computing devices to execute the determined transfer. ## Claims ### Claim 1 A method for memory error recovery comprising: receiving, by a monitoring agent, an indication of a memory error generated in response to a write instruction at a virtual machine (VM) of a computing system; and transmitting, by the monitoring agent, an instruction to a scheduler of the computing system to initiate migration of the VM in response to the memory error generated in response to the write instruction. ### Claim 2 (depends on 1) The method of claim 1, wherein the indication of the memory error is a corrected machine check interrupt (CMCI) signal. ### Claim 3 (depends on 2) The method of claim 2, further comprising: determining, by the monitoring agent, that the CMCI signal is associated with an uncorrectable error, wherein transmitting the instruction to the scheduler is in response to the determination that the CMCI signal is associated with the uncorrectable error. ### Claim 4 (depends on 3) The method of claim 3, wherein the monitoring agent determines that the CMCI signal is associated with the uncorrectable error and transmits the instruction to the scheduler on an order of milliseconds. ### Claim 5 (depends on 1) The method of claim 1, wherein the monitoring agent transmits the instruction to the scheduler prior to a read instruction being executed at the VM. ### Claim 6 (depends on 1) The method of claim 1, further comprising migrating, by one or more processors, the VM from a source machine to a target machine according to a migration instruction from the scheduler. ### Claim 7 (depends on 6) The method of claim 6, wherein migrating the VM from the source machine to the target machine comprises: copying memory associated with the source machine to the target machine; detecting, during the copying, the memory error; and injecting a software recoverable action optional (SRAO) machine check exception (MCE) into the copied memory at a memory page containing the memory error, whereby the memory page containing the memory error is isolated. ### Claim 8 (depends on 7) The method of claim 7, wherein detecting the memory error and injecting the SRAO MCE are performed by a live migration pre-copy thread. ### Claim 9 (depends on 7) The method of claim 7, wherein the SRAO MCE is injected to a single virtual processor core of the computing system. ### Claim 10 (depends on 6) The method of claim 6, wherein migrating the VM from the source machine to the target machine comprises: copying memory associated with the source machine to the target machine; and determining whether a memory page containing the memory error is in use by one or more applications; and in response to determining that the memory page is in use, setting a page fault, such that an attempt by the one or more applications to access the memory page avoids an MCE. ### Claim 11 (depends on 10) The method of claim 10, further comprising, in response to determining that the memory page is not in use, unmapping the memory page, such that the memory page is invisible to the one or more applications. ### Claim 12 A system for memory error recovery comprising one or more processors; and memory in communication with the one or more processors, wherein the memory contains instructions configured to cause the one or more processors to: perform write error monitoring of data being written to a VM of the system; and perform migration of the VM from a source machine to a target machine in response to an uncorrected error with no action (UCNA) detected by the write error monitoring, wherein the UCNA is a memory error. ### Claim 13 (depends on 12) A system of claim 12, wherein the error monitoring includes: receiving CMCI signaling; and interpreting the CMCI signaling as the UNCA. ### Claim 14 (depends on 12) The system of claim 12, wherein the write_error monitoring occurs on an order of milliseconds. ### Claim 15 (depends on 12) The system of claim 12, wherein the instructions are configured to cause the one or more processors to perform migration of the VM by transmitting an instruction to a scheduler. ### Claim 16 (depends on 12) The system of claim 12, wherein the instructions are further configured to cause the one or more processors to: copy memory associated with the source machine to the target machine; detect, during a read operation of the copying, the memory error; and inject an SRAO MCE into the copied memory at a memory page containing the memory error. ### Claim 17 (depends on 16) The system of claim 16, wherein the instructions are configured to cause the one or more processors to inject the SRAO MCE into a single virtual processor core of the target machine. ### Claim 18 (depends on 12) The system of claim 12, wherein the instructions are further configured to cause the one or more processors to: copy the memory error to a memory page of the target machine; and either (i) set a page fault, such that an attempt by one or more applications to access the memory page avoids an MCE; or (ii) unmap the memory page, such that the memory page is invisible to the one or more applications. ### Claim 19 (depends on 18) The system of claim 18, wherein the instructions are further configured to cause the one or more processors to: determine whether a memory page to which the memory error is copied is in use by one or more applications; and in response to a determination that the memory page is in use, set the page fault, such that the attempt by the one or more applications to access the memory page avoids the MCE. ### Claim 20 (depends on 18) The system of claim 18, wherein the instructions are further configured to cause the one or more processors to: determine whether a memory page to which the memory error is copied is in use by one or more applications; and in response to a determination that the memory page is not in use, unmap the memory page, such that the memory page is invisible to the one or more applications.