# US 2021/0334236 A1 · Supporting distributed and local objects using a multi-writer log-structured file system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334236 A1
- Document ID: US20210334236A1
- Status: Published application
- Kind: A1
- Application number: 16/857,517
- Filed: 2020-04-24
- Published: 2021-10-28
- Earliest claimed: 2020-04-24
- Applicants: VMware, Inc.
- Inventors: Wenguang Wang; Vamsi Gunturu
- Web page: https://patentgrep.com/patent/US20210334236A1

## CPC

- G06F 16/11: File system administration, e.g. details of archiving or snapshots (error detection or correction of the data by redundancy in operations G06F11/14)
- G06F 16/1805: {Append-only file systems, e.g. using logs or journals to store data}
- G06F 12/10: Address translation
- G06F 16/182: Distributed file systems
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Supporting distributed and local objects using a multi-writer log-structured file system (LFS) includes, on a node, receiving incoming data from each of a plurality of local objects; coalescing the received data; determining whether the coalesced data comprises a full segment of data; based at least on the coalesced incoming data comprises a full segment, writing at least a first portion of the coalesced data to a first storage of the LFS, wherein the coalesced data comprises the first portion and a remainder portion; writing the remainder portion to a second storage of the LFS; acknowledging the writing to the objects; determining whether at least a full segment of data has accumulated in the second storage; based at least on determining that at least a full segment has accumulated, writing at least a portion of the accumulated data as one or more full segments of data to the first storage.

## Claims

### Claim 1

A method of supporting distributed and local objects using a multi-writer log-structured file system (LFS), the method comprising:
on a first node, receiving incoming data from each of a plurality of objects local to the first node, wherein the plurality of objects is configured to simultaneously write to the multi-writer LFS;
coalescing the received incoming data;
determining whether the coalesced incoming data comprises at least a full segment of data;
based at least on determining that the coalesced incoming data comprises at least a full segment of data, writing at least a first portion of the coalesced incoming data as one or more full segments of data to a first storage of the multi-writer LFS, wherein the coalesced incoming data comprises the first portion and a remainder portion;
writing the remainder portion of the coalesced incoming data to a second storage of the multi-writer LFS;
acknowledging the writing to the plurality of objects;
determining whether at least a full segment of data has accumulated in the second storage; and
based at least on determining that at least a full segment of data has accumulated in the second storage, writing at least a portion of the accumulated data in the second storage as one or more full segments of data to the first storage.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
requesting allocation of new segments of the first storage, wherein the request indicates a local or a global attribute.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
performing segment cleaning processes locally on each of the first node and a second node, the first node being a different physical node from the second node.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
based at least on writing data to the first storage, updating a local segment usage table (SUT) to mark used segments as no longer free;
based at least on performing a segment cleaning process, updating the local SUT to mark freed segments as free; and
merging local SUT updates into a master SUT.

### Claim 5 (depends on 1)

The method of claim 1, wherein writing the remainder portion to the second storage comprises writing the remainder portion to a log; and
the method further comprises, based at least on an object of the plurality of objects moving to a second node, prior to accepting new incoming data from the object, replaying the log to reconstruct a new object map for the object on the second node, wherein the first node is a different physical node from the second node.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
based at least on writing data to the second storage, updating an object map to indicate the writing of the data, wherein a logical-to-physical storage map uses an object identifier as a major key, thereby preventing overlap of object maps; and
based at least on at least writing the portion of the accumulated data to the first storage, updating the object map to indicate the writing of the portion of the accumulated data to the first storage.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first storage comprises a capacity tier, wherein the second storage comprises a performance tier, and wherein writing the remainder portion to the second storage comprises mirroring the remainder portion.

### Claim 8

A computer system for supporting distributed and local objects using a multi-writer log-structured file system (LFS), the computer system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code for transferring data to another computer system, the program code causing the processor to:
on a first node, receive incoming data from each of a plurality of objects local to the first node, wherein the plurality of objects is configured to simultaneously write to the multi-writer LFS;
coalesce the received incoming data;
determine whether the coalesced incoming data comprises at least a full segment of data;
based at least on determining that the coalesced incoming data comprises at least a full segment of data, write at least a first portion of the coalesced incoming data as one or more full segments of data to a first storage of the multi-writer LFS, wherein the coalesced incoming data comprises the first portion and a remainder portion;
write the remainder portion of the coalesced incoming data to a second storage of the multi-writer LFS;
acknowledge the writing to the plurality of objects;
determine whether at least a full segment of data has accumulated in the second storage; and
based at least on determining that at least a full segment of data has accumulated in the second storage, write at least a portion of the accumulated data in the second storage as one or more full segments of data to the first storage.

### Claim 9 (depends on 8)

The computer system of claim 8, wherein the program code is further operative to:
request allocation of new segments of the first storage, wherein the request indicates a local or a global attribute.

### Claim 10 (depends on 8)

The computer system of claim 8, wherein the program code is further operative to:
perform segment cleaning processes locally on each of the first node and a second node, the first node being a different physical node from the second node.

### Claim 11 (depends on 8)

The computer system of claim 8, wherein the program code is further operative to:
based at least on writing data to the first storage, update a local segment usage table (SUT) to mark used segments as no longer free;
based at least on performing a segment cleaning process, update the local SUT to mark freed segments as free; and
merge local SUT updates into a master SUT.

### Claim 12 (depends on 8)

The computer system of claim 8, wherein writing the remainder portion to the second storage comprises writing the remainder portion to a log; and
wherein the program code is further operative to, based at least on an object of the plurality of objects moving to a second node, prior to accepting new incoming data from the object, replay the log to reconstruct a new object map for the object on the second node, wherein the first node is a different physical node from the second node.

### Claim 13 (depends on 8)

The computer system of claim 8, wherein the program code is further operative to:
based at least on writing data to the second storage, update an object map to indicate the writing of the data, wherein a logical-to-physical storage map uses an object identifier as a major key, thereby preventing overlap of object maps; and
based at least on at least writing the portion of the accumulated data to the first storage, update the object map to indicate the writing of the portion of the accumulated data to the first storage.

### Claim 14 (depends on 8)

The computer system of claim 8, wherein the first storage comprises a capacity tier, wherein the second storage comprises a performance tier, and wherein writing the remainder portion to the second storage comprises mirroring the remainder portion.

### Claim 15

A non-transitory computer readable storage medium having stored thereon program code executable by a first computer system at a first site, the program code embodying a method comprising:
on a first node, receiving incoming data from each of a plurality of objects local to the first node, wherein the plurality of objects is configured to simultaneously write to the multi-writer LFS;
coalescing the received incoming data;
determining whether the coalesced incoming data comprises at least a full segment of data;
based at least on determining that the coalesced incoming data comprises at least a full segment of data, writing at least a first portion of the coalesced incoming data as one or more full segments of data to a first storage of the multi-writer LFS, wherein the coalesced incoming data comprises the first portion and a remainder portion;
writing the remainder portion of the coalesced incoming data to a second storage of the multi-writer LFS;
acknowledging the writing to the plurality of objects;
determining whether at least a full segment of data has accumulated in the second storage; and
based at least on determining that at least a full segment of data has accumulated in the second storage, writing at least a portion of the accumulated data in the second storage as one or more full segments of data to the first storage.

### Claim 16 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the program code further comprises:
requesting allocation of new segments of the first storage, wherein the request indicates a local or a global attribute.

### Claim 17 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the program code further comprises:
performing segment cleaning processes locally on each of the first node and a second node, the first node being a different physical node from the second node.

### Claim 18 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the program code further comprises:
based at least on writing data to the first storage, updating a local segment usage table (SUT) to mark used segments as no longer free;
based at least on performing a segment cleaning process, updating the local SUT to mark freed segments as free; and
merging local SUT updates into a master SUT.

### Claim 19 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein writing the remainder portion to the second storage comprises writing the remainder portion to a log; and
wherein the program code further comprises, based at least on an object of the plurality of objects moving to a second node, prior to accepting new incoming data from the object, replaying the log to reconstruct a new object map for the object on the second node, wherein the first node is a different physical node from the second node.

### Claim 20 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the first storage comprises a capacity tier, wherein the second storage comprises a performance tier, and wherein writing the remainder portion to the second storage comprises mirroring the remainder portion.
