# US 10,769,036 B2 · Distributed transaction log > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,036 B2 - Document ID: US10769036B2 - Status: Granted - Kind: B2 - Application number: 15/810,650 - Filed: 2017-11-13 - Published: 2018-03-08 - Granted: 2020-09-08 - Earliest claimed: 2013-08-26 - Assignees: VMware, Inc. - Applicants: VMware, Inc. - Inventors: William Earl; Christos Karamanolis; Eric Knauft; Pascal Renauld - Web page: https://patentgrep.com/patent/US10769036B2 ## CPC - G06F 11/1471: involving logging of persistent data for recovery - G06F 16/1774: {Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files} - G06F 16/1815: {Journaling file systems} - G06F 16/2365: {Ensuring data consistency and integrity} - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Embodiments of the disclosure provide techniques for updating a distributed transaction log on a previously offline resource object component using distributed transaction logs from active host computer nodes from separate RAID mirror configurations. Each component object maintains a journal (log) where distributed transactions are recorded. If a component object goes offline and subsequently returns (e.g., if the node hosting the component object reboots), the component object is marked as stale. To return the component object to an active state, a distributed resources module retrieves the journals from other resource component objects from other RAID configurations where the data is mirrored. The module filters corresponding data that is missing in the journal of the previously offline corresponding object and merges the filtered data to the journal. ## Claims ### Claim 1 A computer-implemented method comprising: receiving an input/output (I/O) operation request from a virtual machine at a hypervisor of a host computer node, the I/O operation request for a corresponding virtual disk of the virtual machine, wherein the virtual disk is associated with a distributed storage system; determining particular resource component objects of the virtual disk that are subject to the I/O operation, wherein each resource component object is associated with particular physical storage resources of the distributed storage system; performing a plurality of distributed transactions on the particular resource component objects to carry out the I/O operation; logging each transaction in a journal for each respective resource component object, wherein each entry includes a sequence identifier; and reconciling the entries on different journals such that lost transactions from an offline resource component object are derived from comparing the content of the journals using the respective sequence identifiers. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the I/O request includes an identifier for an object representing the virtual disk, wherein the hypervisor uses the identifier to identify a coordinator node of the virtual disk object for carrying out the I/O request. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the particular resource component objects are hosted on one or more host computer nodes in a cluster. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein active resource component objects of the particular resource component objects correspond to a mirrored redundant array of independent disks (RAID) configuration. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: determining that one of the particular resource component objects is stale; and using the content of the journals to resynchronize the stale resource component object. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein using the content of the journals to resynchronize the state resource component object includes determining missing transactions for the stale resource component object using the sequence identifiers and applying any corresponding transactions to the state resource component object. ### Claim 7 A system comprising: one or more computers and one or more storage devices on which are stored data that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving an input/output (I/O) operation request from a virtual machine at a hypervisor, the I/O operation request for a corresponding virtual disk of the virtual machine, wherein the virtual disk is associated with a distributed storage system; determining particular resource component objects of the virtual disk that are subject to the I/O operation, wherein each resource component object is associated with particular physical storage resources of the distributed storage system; performing a plurality of distributed transactions on the particular resource component objects to carry out the I/O operation; logging each transaction in a journal for each respective resource component object, wherein each entry includes a sequence identifier; and reconciling the entries on different journals such that lost transactions from an offline resource component object are derived from comparing the content of the journals using the respective sequence identifiers. ### Claim 8 (depends on 7) The system of claim 7, wherein the I/O request includes an identifier for an object representing the virtual disk, wherein the hypervisor uses the identifier to identify a coordinator node of the virtual disk object for carrying out the I/O request. ### Claim 9 (depends on 7) The system of claim 7, wherein the particular resource component objects are hosted on one or more host computer nodes in a cluster. ### Claim 10 (depends on 7) The system of claim 7, wherein active resource component objects of the particular resource component objects correspond to a mirrored redundant array of independent disks (RAID) configuration. ### Claim 11 (depends on 7) The system of claim 7, further comprising: determining that one of the particular resource component objects is stale; and using the content of the journals to resynchronize the stale resource component object. ### Claim 12 (depends on 11) The system of claim 11, wherein using the content of the journals to resynchronize the state resource component object includes determining missing transactions for the stale resource component object using the sequence identifiers and applying any corresponding transactions to the state resource component object. ### Claim 13 A non-transitory computer-readable storage medium storing instructions, which, when executed on a processor, performs operations comprising: receiving an input/output (I/O) operation request from a virtual machine at a hypervisor, the I/O operation request for a corresponding virtual disk of the virtual machine, wherein the virtual disk is associated with a distributed storage system; determining particular resource component objects of the virtual disk that are subject to the I/O operation, wherein each resource component object is associated with particular physical storage resources of the distributed storage system; performing a plurality of distributed transactions on the particular resource component objects to carry out the I/O operation; logging each transaction in a journal for each respective resource component object, wherein each entry includes a sequence identifier; and reconciling the entries on different journals such that lost transactions from an offline resource component object are derived from comparing the content of the journals using the respective sequence identifiers. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the I/O request includes an identifier for an object representing the virtual disk, wherein the hypervisor uses the identifier to identify a coordinator node of the virtual disk object for carrying out the I/O request. ### Claim 15 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein the particular resource component objects are hosted on one or more host computer nodes in a cluster. ### Claim 16 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein active resource component objects of the particular resource component objects correspond to a mirrored redundant array of independent disks (RAID) configuration. ### Claim 17 (depends on 13) The non-transitory computer-readable storage medium of claim 13, further comprising: determining that one of the particular resource component objects is stale; and using the content of the journals to resynchronize the stale resource component object. ### Claim 18 (depends on 17) The non-transitory computer-readable storage medium of claim 17, wherein using the content of the journals to resynchronize the state resource component object includes determining missing transactions for the stale resource component object using the sequence identifiers and applying any corresponding transactions to the state resource component object. ### Claim 19 A system comprising: one or more computers and one or more storage devices on which are stored data that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: preventing previously offline resource component objects in a distributed system from servicing operations if the resource component objects are stale, the system performing transactions without the resource component objects, the resource component objects to be resynchronized from other online resource component objects, comprising: when a resource component object goes offline and the system makes progress without it, persisting a record of the fact that it is offline on a majority of all online resource component objects, said majority being a majority of the total number of resource components; maintaining information of which resource component objects are stale in the running system and using this information in deciding which resource component objects must participate in transactions; upon regaining connectivity to the previously offline resource component objects, disallowing the stale resource component object from servicing operations until a resynchronization method is complete, wherein the resynchronization comprises obtaining transaction logs from one or more other online resource component objects, each transaction log including entries identifying actions performed on each of the one or more online resource component objects for each distributed transaction of the distributed system, and updating the resource component object based on the transaction logs; and upon completion of the resynchronization, persisting on a majority of nodes the fact that the resource component object is no longer stale before allowing it to participate in transactions. ### Claim 20 (depends on 19) The system of claim 19, wherein a record that a resource component object is stale is persisted before any pending or new transactions that would otherwise require the offline resource component object are allowed to proceed. ### Claim 21 (depends on 20) The system of claim 20, wherein data is persisted in the form of a configuration containing state for each resource component object, the state indicating at least one of an active state or a stale state.