US 10,474,636 B2Granted

Block allocation for low latency file systems

AssigneeAmazon Technologies, Inc.

InventorsJacob A. Strauss, Michael Robert Frasca, Neal John Charbonneau

Life of this patent

Estimating the term…

Loading patent…

Abstract

An LL server (LLS) may process metadata requests for a file system in LL mode in a distributed file storage services (DFSS). For requests that require allocating blocks to file system objects in the backing store, instead of relying on distributed transactions used for file systems in high throughput (HT) mode, a pool of blocks may be pre-allocated for the LL file system in the backing store, and a free block list may be maintained in local memory of the LLS. When a metadata operation requires blocks to be allocated, the blocks are assigned to the respective object from the blocks in the pool. A background process may allocate new blocks in the pool upon the number of blocks dropping below a threshold, or upon block allocation rate for the file system increasing.

Claims20

Dependent claims

A distributed storage service, comprising:

a plurality of computing devices that implement:
a storage subsystem configured to store data and metadata for a plurality of file systems;
a file server associated with one of the plurality of file systems, wherein the file server maintains a metadata journal;
wherein the file server is configured to, in a first mode:
pre-allocate blocks in the storage subsystem for the file system, wherein the blocks are placed in a block pool maintained for the file system in the storage subsystem indicating respective block identifiers of the blocks in the storage subsystem, and the blocks are allocated to the file system but not assigned to a particular file system object in the file system;
receive a request for the file system, wherein the request indicates a file system operation that requires allocation of one or more new blocks for a file system object in the storage subsystem;
obtain block identifiers for one or more blocks from the block pool;
write one or more journal entries to the metadata journal specifying the file system operation and the block identifiers for the one or more blocks;
send a response message indicating that the file system operation has completed and including the block identifiers for the one or more blocks;
and
commit file system operation specified by the one or more journal entries in the metadata journal to the storage subsystem subsequent to sending the response message, wherein said commit comprises assigning the one or more blocks to the file system object in the storage sub system;
wherein file system data is written to at least one of the one or more blocks indicated by the block identifiers prior to said commit.

9 more elements

A non-transitory computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to implement:

a file server associated with a respective one of a plurality of file systems in a distributed storage service, wherein the file server is configured to:
pre-allocate, blocks in a storage system, wherein the blocks are placed in a block pool in the storage system indicating respective block identifiers of the blocks in the storage system, and the blocks are allocated to the file system but not assigned to a particular file system object in the file system;
receive a metadata request for the file system, wherein the metadata request indicates a file system operation that requires allocation of one or more new blocks for a file system object in the storage system;
obtain block identifiers for one or more blocks from the block pool;
write one or more journal entries to a metadata journal for the file system maintained in the storage system, the one or more journal entries specifying the file system operation and the block identifiers for the one or more blocks;
send a response message indicating that the file system operation has completed and including the block identifiers for the one or more blocks;
and
commit the file system operation specified by the one or more journal entries in the metadata journal to the storage system subsequent to sending the response message, wherein said commit comprises assigning the one or more blocks to the file system object.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/081,652Filed 25 Mar 2016
Published: US 2017/0277709 A128 Sep 2017
Granted: 12 Nov 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification9 codes

G06F 16/1727
Details of further file system functions: details of free space management performed by the file system
G06F 3/0604
Specifically adapted to achieve a particular effect: improving or facilitating administration, e.g. storage management
G06F 3/0613
Improving I/O performance: in relation to throughput
G06F 3/0631
Configuration or reconfiguration of storage systems: by allocating resources to storage systems