US 9,710,535 B2Granted

Object storage system with local transaction logs, a distributed namespace, and optimized support for user directories

AssigneeNexenta Systems, Inc.

InventorsAlexander Aizman, Caitlin Bestler, Robert E. Novak

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present invention relates to object storage systems that support hierarchical directories within a namespace manifest stored as an object. The namespace manifest is stored as namespace manifest shards that are determined based on a partial key derived from the full name of the referenced version manifest in each directory entry. Each storage server maintains a local transaction log that keeps track of changes to the namespace manifest in response to put transactions. The namespace manifest can be updated after each put transaction is acknowledged, which minimizes the number of write operations required for a put transaction. Updates to the namespace manifest are batched, distributed, and processed concurrently and in parallel by the storage servers that store the corresponding shards. This reduces the latency that would otherwise exist in the object storage cluster as a result of a put transaction. Updates to namespace manifest shards optionally occur in a batch process using MapReduce techniques.

Claims7

Dependent claims

A method for a storage server within an object storage system to process a put transaction for a new version of an object associated with a new version manifest chunk, wherein the storage server contains an old version of the object associated with an old version manifest chunk, the method comprising:

validating the new version manifest chunk;
saving the new version manifest chunk to persistent storage;
comparing a unique version identifier for the new version manifest with a unique version identifier for the old version manifest stored in a local name index and updating a local name index for the object's name if the unique version identifier for the new version manifest is more current than the unique version identifier for the old version manifest stored in the local name index;
creating an entry in the storage server's local transaction log recording the existence of the new version manifest;
sending an acknowledgement message for the put transaction;
and
asynchronously processing the entry to update one or more namespace manifests of the object storage system;
wherein each transaction log entry specifying creation or deletion of version manifests is mapped into records to be distributed to namespace manifests, wherein the mapped records comprise inclusive mapped records and iterative mapped records;
and
wherein the inclusive mapped records each comprises type, enclosing directory name, the remaining portion of the fully qualified object name which is not part of the enclosing directory name, unique version identifier, and cryptographic hash identifying token of the version manifest, wherein the type comprises data indicating a version manifest exists or data indicating a version manifest does not exist.

7 more elements

validating the new version manifest chunk;
saving the new version manifest chunk to persistent storage;
comparing a unique version identifier for the new version manifest with a unique version identifier for the old version manifest stored in a local name index and updating a local name index for the object's name if the unique version identifier for the new version manifest is more current than the unique version identifier for the old version manifest stored in the local name index;
creating an entry in the storage server's local transaction log recording the existence of the new version manifest;
sending an acknowledgement message for the put transaction;
and
asynchronously processing the entry to update one or more namespace manifests of the object storage system;
wherein each transaction log entry specifying creation or deletion of version manifests is mapped into records to be distributed to namespace manifests, wherein the mapped records comprise inclusive mapped records and iterative mapped records;
and
wherein each mapped record is assigned to a specific namespace manifest by a configuration rule assigning subsets of the entire namespace to specific namespace manifests, and to a specific shard of that namespace manifest, by the cryptographic hash of the enclosing path name.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/820,471Filed 6 Aug 2015 · continuation in part
Priority: 12 Aug 2011
Published: US 2015/0347553 A13 Dec 2015
Granted: 18 Jul 2017
Term ends: Not estimatedThe source records don’t support an estimate.

Classification9 codes

G06F 16/273
Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system…:…
G06F 16/1752
De-duplication implemented within the file system, e.g. based on file segments: based on file chunks
G06F 16/178
Details of further file system functions: techniques for file synchronisation in file systems
G06F 16/1844
Implemented as replicated file system: management specifically adapted to replicated file systems

Family

Application13/029,342Parent of this continuation-in-part · filed 12 Aug 2011
Application13/415,742Parent of this continuation-in-part · filed 8 Mar 2012
Application13/624,593Parent of this continuation · filed 21 Sep 2012
Application14/095,839Parent of this continuation-in-part · filed 3 Dec 2013
Application14/095,843Parent of this continuation-in-part · filed 3 Dec 2013
Application14/095,848Parent of this continuation-in-part · filed 3 Dec 2013
Application14/095,855Parent of this continuation-in-part · filed 3 Dec 2013
Application14/258,791Parent of this continuation-in-part · filed 22 Apr 2014
Application62/040,962Us provisional application · filed 22 Aug 2014
Application62/098,727Us provisional application · filed 31 Dec 2014
US 2015/0347553 A1Pre-grant publication · published 3 Dec 2015
US 9,507,812 B2Parent of this continuation-in-part
US 8,745,095 B2Parent of this continuation
Application14/258,791Parent of this continuation
US 8,533,231 B2Parent of this continuation-in-part
Application13/624,593Parent of this continuation-in-part
US 8,849,759 B2Parent of this continuation-in-part
US 9,338,019 B2Parent of this continuation-in-part
US 9,344,287 B2Parent of this continuation-in-part
US 9,385,874 B2Parent of this continuation-in-part
US 9,385,875 B2Parent of this continuation-in-part
US 2011/0222812 A1Published application
US 2013/0185258 A1Published application
US 2013/0226978 A1Published application
US 2014/0204940 A1Published application
US 2014/0204941 A1Published application
US 2014/0207849 A1Published application
US 2014/0207850 A1Published application
US 2014/0229455 A1Published application
US 8,363,986 B2Related patent
US 9,710,535 B2This patent

From the US filing history. Foreign family members aren’t listed.