# US 8,544,004 B2 · Cluster-based operating system-agnostic virtual computing system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,544,004 B2 - Document ID: US8544004B2 - Status: Granted - Kind: B2 - Application number: 13/290,087 - Filed: 2011-11-06 - Published: 2012-03-01 - Granted: 2013-09-24 - Earliest claimed: 2003-08-11 - Assignees: Scalemp Inc. - Applicants: Shai Fultheim; Herb Zlotogorski; Yaniv Romem - Inventors: Shai Fultheim; Herb Zlotogorski; Yaniv Romem - Web page: https://patentgrep.com/patent/US8544004B2 ## CPC - G06F 9/45533: {Hypervisors; Virtual machine monitors} - G06F 9/4411: {Configuring for operating with peripheral devices; Loading of device drivers} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract According to a disclosed embodiment of the invention, an improved cluster-based collection of computers (nodes) is realized using conventional computer hardware. Software is provided that enables at least one virtual machine to be presented to guest operating systems, wherein each node participating with the virtual machine has its own emulator or virtual machine monitor. VM memory coherency and I/O coherency are provided by hooks, which result in the manipulation of internal processor structures. A private network provides communication among the nodes. ## Claims ### Claim 1 A computer system, comprising: a cluster of computing nodes, each computing node comprising: a memory; a network interface device, which connects the computing node to a network interconnecting the computing nodes in the cluster; and at least one CPU, which is configured to run a respective virtual machine implementer program on the computing node, while communicating via the network interface device with other computing nodes in the cluster, wherein the other computing nodes run respective virtual machine implementer programs, so that the respective virtual machine implementer programs jointly support a shared virtual machine implemented concurrently on two or more of the computing nodes, wherein a guest operating system runs on the shared virtual machine, and instructions of the guest operating system are distributed for execution among the nodes as though the system were a single symmetric multiprocessing machine with shared memory. ### Claim 2 (depends on 1) The system according to claim 1, wherein the computing nodes are configured to support the shared virtual machine so that a software application running over the guest operating system need not be cluster-aware. ### Claim 3 (depends on 1) The system according to claim 1, wherein each of the computing nodes comprises multiple CPUs, which run the virtual machine implementer programs. ### Claim 4 (depends on 1) The system according to claim 1, wherein the shared virtual machine includes a plurality of virtual CPUs, which are controlled by the guest operating system. ### Claim 5 (depends on 1) The system according to claim 1, wherein the virtual machine implementer program on at least one of the nodes comprises multiple virtual machine implementer programs, which are configured to provide multiple, respective virtual nodes, and wherein the virtual machine is supported by the virtual nodes. ### Claim 6 (depends on 1) The system according to claim 1, wherein the computing nodes comprise physical peripheral component interconnect controllers, and wherein the virtual machine implementer programs comprise a virtual peripheral component interconnect controller, which serves as a proxy between the guest operating system and the physical peripheral component interconnect controllers. ### Claim 7 (depends on 6) The system according to claim 6, wherein the virtual peripheral component interconnect controller is configured to emulate hot-pluggable events for the guest operating system, so as to allow dynamic addition and removal of one or more of the nodes while running the shared virtual machine and the guest operating system. ### Claim 8 (depends on 1) The system according to claim 1, wherein the virtual machine implementer programs maintain mirrors of the memory used by the guest operating system in each of the nodes, and are configured to write-invalidate at least a portion of a page of the memory in one of the nodes and to transfer a valid copy of the portion of the page to the one of the nodes from another of the nodes via the network. ### Claim 9 A method for computing, comprising: providing a cluster of computing nodes interconnected by a network, each computing node comprising at least one CPU, a memory, and a network interface device, which connects the computing node to the network interconnecting the plurality of the computing nodes; and running respective virtual machine implementer programs on the at least one CPU in two or more of the computing nodes, while communicating among the two or more of the computing nodes via the network interface device, wherein the respective virtual machine implementer programs jointly support a shared virtual machine implemented concurrently on the two or more of the computing nodes, wherein a guest operating system runs on the shared virtual machine, and instructions of the guest operating system are distributed for execution among the nodes as though the system were a single symmetric multiprocessing machine with shared memory. ### Claim 10 (depends on 9) The method according to claim 9, wherein running the respective virtual machine implementer programs comprises supporting the shared virtual machine so that a software application running over the guest operating system need not be cluster-aware. ### Claim 11 (depends on 9) The method according to claim 9, wherein running the respective virtual machine implementer programs comprises running virtual machine implementer programs on multiple CPUs in each of the two or more of the computing nodes. ### Claim 12 (depends on 9) The method according to claim 9, wherein the shared virtual machine includes a plurality of virtual CPUs, which are controlled by the guest operating system. ### Claim 13 (depends on 9) The method according to claim 9, wherein running the respective virtual machine implementer programs comprises running, on at least one of the nodes, multiple virtual machine implementer programs, which are configured to provide multiple, respective virtual nodes, and wherein the virtual machine is supported by the virtual nodes. ### Claim 14 (depends on 9) The method according to claim 9, wherein the computing nodes comprise physical peripheral component interconnect controllers, and wherein running the respective virtual machine implementer programs comprises running a virtual peripheral component interconnect controller, which serves as a proxy between the guest operating system and the physical peripheral component interconnect controllers. ### Claim 15 (depends on 14) The method according to claim 14, and comprising emulating, using the virtual peripheral component interconnect controller, hot-pluggable events for the guest operating system, so as to allow dynamic addition and removal of one or more of the nodes while running the shared virtual machine and the guest operating system. ### Claim 16 (depends on 9) The method according to claim 9, and comprising, using the virtual machine implementer programs, maintaining mirrors of the memory used by the guest operating system in each of the nodes, write-invalidating at least a portion of a page of the memory in one of the nodes, and transferring a valid copy of the portion of the page to the one of the nodes from another of the nodes via the network. ### Claim 17 A computer software product, comprising a non-transitory computer-readable medium in which program instructions are stored, which instructions are configured to run on a cluster of computing nodes, each computing node including a memory, a network interface device, which connects the computing node to a network interconnecting the computing nodes in the cluster, and at least one CPU, wherein the instructions cause the at least one CPU to run a respective virtual machine implementer program on the computing node, while communicating via the network interface device with other computing nodes in the cluster, wherein the other computing nodes run respective virtual machine implementer programs, so that the respective virtual machine implementer programs jointly support a shared virtual machine implemented concurrently on two or more of the computing nodes, wherein a guest operating system runs on the shared virtual machine, and instructions of the guest operating system are distributed for execution among the nodes as though the system were a single symmetric multiprocessing machine with shared memory. ### Claim 18 (depends on 17) The product according to claim 17, wherein the instructions cause the computing nodes to support the shared virtual machine so that a software application running over the guest operating system need not be cluster-aware. ### Claim 19 (depends on 17) The product according to claim 17, wherein each of the computing nodes include multiple CPUs, and wherein the instructions cause the multiple CPUs to run the virtual machine implementer programs. ### Claim 20 (depends on 17) The product according to claim 17, wherein the shared virtual machine includes a plurality of virtual CPUs, which are controlled by the guest operating system. ### Claim 21 (depends on 17) The product according to claim 17, wherein the virtual machine implementer program on at least one of the computing nodes comprises multiple virtual machine implementer programs and causes the computing nodes to provide multiple, respective virtual nodes, and wherein the virtual machine is supported by the virtual nodes. ### Claim 22 (depends on 17) The product according to claim 17, wherein the computing nodes include physical peripheral component interconnect controllers, and wherein the instructions cause the computing nodes to provide a virtual peripheral component interconnect controller, which serves as a proxy between the guest operating system and the physical peripheral component interconnect controllers. ### Claim 23 (depends on 17) The product according to claim 17, wherein the virtual peripheral component interconnect controller is configured to emulate hot-pluggable events for the guest operating system, so as to allow dynamic addition and removal of one or more of the nodes while running the shared virtual machine and the guest operating system. ### Claim 24 (depends on 17) The product according to claim 17, wherein the instructions cause the computing nodes to maintain mirrors of the memory used by the guest operating system in each of the nodes, and to write-invalidate at least a portion of a page of the memory in one of the nodes and to transfer a valid copy of the portion of the page to the one of the nodes from another of the nodes via the network. ### Claim 25 A computer system, comprising: a cluster of computing nodes, each computing node comprising: a physical memory; a network interface device, which connects the computing node to a network interconnecting the computing nodes in the cluster; and at least one CPU, which is configured to run a respective virtual machine implementer program on the computing node, while communicating via the network interface device with other computing nodes in the cluster, wherein the other computing nodes run respective virtual machine implementer programs, so that the respective virtual machine implementer programs jointly support a shared virtual machine implemented concurrently on two or more of the computing nodes, wherein a guest operating system runs on the shared virtual machine, which presents a single coherent shared memory to the guest operating system, using the physical memory that is distributed across multiple nodes. ### Claim 26 (depends on 25) The system according to claim 25, wherein the computing nodes are configured to support the shared virtual machine so that a software application running over the guest operating system need not be cluster-aware. ### Claim 27 (depends on 25) The system according to claim 25, wherein the computing nodes comprise physical peripheral component interconnect controllers, and wherein the virtual machine implementer programs comprise a virtual peripheral component interconnect controller, which serves as a proxy between the guest operating system and the physical peripheral component interconnect controllers. ### Claim 28 (depends on 25) The system according to claim 25, wherein the virtual machine implementer programs maintain mirrors of the memory used by the guest operating system in each of the nodes, and are configured to write-invalidate at least a portion of a page of the memory in one of the nodes and to transfer a valid copy of the portion of the page to the one of the nodes from another of the nodes via the network.