# US 2021/0240356 A1 · Handling frequently accessed pages > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240356 A1 - Document ID: US20210240356A1 - Status: Published application - Kind: A1 - Application number: 17/236,952 - Filed: 2021-04-21 - Published: 2021-08-05 - Earliest claimed: 2017-06-27 - Assignees: TidalScale, Inc. - Applicants: TidalScale, Inc. - Inventors: Isaac R. Nassi; Kleoni Ioannidou; Michael Berman; I-Chun Fang; Mark Hill; Brian Moffet; Jeffrey Paul Radick; David P. Reed; Keith Reynolds - Web page: https://patentgrep.com/patent/US20210240356A1 ## CPC - G06F 3/0611: {in relation to response time} - G06F 3/0647: {Migration mechanisms} - G06F 3/0653: {Monitoring storage devices or systems} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 9/4856: {resumption being on a different machine, e.g. task migration, virtual machine migration (G06F9/5088 takes precedence)} - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 9/5033: {considering data affinity} - G06F 9/5044: {considering hardware capabilities} - G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)} - G06F 11/3006: {where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50)} - G06F 11/3433: {for load management (allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029)} - G06F 11/3452: {Performance evaluation by statistical analysis} - G06F 12/08: in hierarchically structured memory systems, e.g. virtual memory systems - G06F 12/109: for multiple virtual address spaces, e.g. segmentation (G06F12/1036 takes precedence) - G06F 2201/815: Virtual - G06F 2209/509: Offload - G06F 2212/1024: Latency reduction - G06F 2212/151: Emulated environment, e.g. virtual machine - G06F 2212/152: Virtualized environment, e.g. logically partitioned system - G06F 2212/502: using adaptive policy - G06F 2212/651: Multi-level translation tables - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Handling frequently accessed pages is disclosed. An indication is received of a stalling event caused by a requested portion of memory being inaccessible. It is determined that the requested portion of memory is a frequently updated portion of memory. The stalling event is handled based at least in part on the determination that the requested portion of memory is a frequently updated portion of memory. ## Claims ### Claim 1 A computer system, comprising: a plurality of physically interconnected computing nodes, wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on the plurality of physically interconnected computing nodes; wherein an indication is received, by a hyper-kernel on a first computing node, of a stalling event caused by a requested portion of memory being inaccessible; wherein it is determined, at least in part by the hyper-kernel performing a query of a data structure comprising identifiers of portions of memory designated as being frequently updated, whether the requested portion of memory is designated as a frequently updated portion of memory; and wherein the stalling event is handled based at least in part on the determination of whether the requested portion of memory is designated as a frequently updated portion of memory. ### Claim 2 (depends on 1) The computer system of claim 1, wherein the query is performed using an address of the requested portion of memory. ### Claim 3 (depends on 2) The computer system of claim 2, wherein the address comprises a guest physical address. ### Claim 4 (depends on 1) The computer system of claim 1, wherein an identifier of a portion of memory is included in the data structure based at least in part on a frequency with which the portion of memory is accessed. ### Claim 5 (depends on 4) The computer system of claim 4, wherein the frequency with which the portion of memory is accessed is determined based at least in part on a number of write stalls that have occurred with respect to the portion of memory. ### Claim 6 (depends on 4) The computer system of claim 4, wherein the identifier of the portion of memory is included in the data structure based at least in part on a determination that the frequency with which the portion of memory is accessed exceeds a threshold. ### Claim 7 (depends on 1) The computer system of claim 1, wherein the stalling event comprises a write stall involving a virtual processor attempting to write into the requested portion of memory. ### Claim 8 (depends on 1) The computer system of claim 1, wherein the requested portion of memory is needed by a virtual processor on the first computing node, wherein the requested portion of memory is on a second computing node, wherein based at least in part on a determination that the requested portion of memory is designated as a frequently updated portion of memory, the stalling event is handled at least in part by migrating the virtual processor to the second computing node to update the requested portion of memory, and wherein the virtual processor is migrated back to the first computing node in response to completion of the update to the requested portion of memory. ### Claim 9 (depends on 8) The computer system of claim 8, wherein copies of the updated page are broadcasted to at least some of the computing nodes in the plurality of physically interconnected computing nodes. ### Claim 10 (depends on 1) The computer system of claim 1, wherein the requested portion of memory is needed by a virtual processor on the first computing node, wherein the requested portion of memory is on a second computing node, wherein based at least in part on a determination that the requested portion of memory is a frequently updated portion of memory, the stalling event is handled at least in part by migrating the requested portion of memory to the first computing node to be updated, and wherein the requested portion of memory is migrated back to the second computing node in response to completion of an update to the requested portion of memory. ### Claim 11 A method, comprising: receiving, by a hyper-kernel on a first computing node, an indication of a stalling event caused by a requested portion of memory being inaccessible, wherein the first computing node is included in a plurality of physically interconnected computing nodes, and wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on the plurality of physically interconnected computing nodes; determining, at least in part by the hyper-kernel performing a query of a data structure comprising identifiers of portions of memory designated as being frequently updated, whether the requested portion of memory is designated as a frequently updated portion of memory; and handling the stalling event based at least in part on the determination of whether the requested portion of memory is designated as a frequently updated portion of memory. ### Claim 12 (depends on 11) The method of claim 11, wherein the query is performed using an address of the requested portion of memory. ### Claim 13 (depends on 12) The method of claim 12, wherein the address comprises a guest physical address. ### Claim 14 (depends on 11) The method of claim 11, wherein an identifier of a portion of memory is included in the data structure based at least in part on a frequency with which the portion of memory is accessed. ### Claim 15 (depends on 14) The method of claim 14, wherein the frequency with which the portion of memory is accessed is determined based at least in part on a number of write stalls that have occurred with respect to the portion of memory. ### Claim 16 (depends on 14) The method of claim 14, wherein the identifier of the portion of memory is included in the data structure based at least in part on a determination that the frequency with which the portion of memory is accessed exceeds a threshold. ### Claim 17 (depends on 11) The method of claim 11, wherein the stalling event comprises a write stall involving a virtual processor attempting to write into the requested portion of memory. ### Claim 18 (depends on 11) The method of claim 11, wherein the requested portion of memory is needed by a virtual processor on the first computing node, wherein the requested portion of memory is on a second computing node, wherein based at least in part on the determination that the requested portion of memory is designated as a frequently updated portion of memory, the stalling event is handled at least in part by migrating the virtual processor to the second computing node to update the requested portion of memory, and wherein the virtual processor is migrated back to the first computing node in response to completion of the update to the requested portion of memory. ### Claim 19 (depends on 18) The method of claim 18, wherein copies of the updated page are broadcasted to at least some of the computing nodes in the plurality of physically interconnected computing nodes. ### Claim 20 (depends on 11) The method of claim 11, wherein the requested portion of memory is needed by a virtual processor on the first computing node, wherein the requested portion of memory is on a second computing node, wherein based at least in part on the determination that the requested portion of memory is a frequently updated portion of memory, the stalling event is handled at least in part by migrating the requested portion of memory to the first computing node to be updated, and wherein the requested portion of memory is migrated back to the second computing node in response to completion of an update to the requested portion of memory. ### Claim 21 A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for: receiving, by a hyper-kernel on a first computing node, an indication of a stalling event caused by a requested portion of memory being inaccessible, wherein the first computing node is included in a plurality of physically interconnected computing nodes, and wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on the plurality of physically interconnected computing nodes; determining, at least in part by the hyper-kernel performing a query of a data structure comprising identifiers of portions of memory designated as being frequently updated, whether the requested portion of memory is designated as a frequently updated portion of memory; and handling the stalling event based at least in part on the determination of whether the requested portion of memory is designated as a frequently updated portion of memory.