File System

A non-transitory machine-readable storage medium encoded with instructions executable by a data processing apparatus to write a file to a system, the machine-readable storage medium comprising instructions to:

• receive the file at a namespace file system of the system,
• wherein the file includes data and has a filename, and
• wherein the system includes the namespace file system and an object store;
• associate the file with an index node (inode) number, wherein the inode number is an integer;
• break the file data into a set of data objects;
• generate a data hash for each data object of the set of data objects to identify and access each data object, wherein the data hash is derived at least from a content of the data object;
• generate a file object for the file that maps the data hashes of the set of data objects of the file;
• generate a file hash for the file object to identify and access each file object, wherein the file hash is derived at least from a content of the file object;
• map the inode number to the file hash;
• store the set of data objects and the file object in the object store;
• and
• generate a transaction log of object activity in the object store.

A non-transitory machine-readable storage medium encoded with instructions executable by a data processing apparatus, including instructions to:

• implement a file system comprising a namespace layer and an object layer, wherein the namespace layer and the object layer are in user-space;
• store data and metadata from the namespace layer in objects in an object store of the object layer,
• store data of a file as a set of data objects in the object store such that there is at most a single copy within the object store of each data object of the set of data objects;
• generate a hash for each data object of the set of data objects that is derived at least from a content of the respective data object and that is used to access the respective data object in the object store;
• store a file object corresponding to the file in the object store, the file object including the hashes of the set of data objects;
• generate a hash for the file object that is derived at least from a content of the file object and that is used to access the file object in the object store;
• store a mapping object in the object store, the mapping object including a mapping of an index node (inode) number to the hash of the file object;
• generate a hash for the mapping object that is derived at least from a content of the respective mapping object and that is used to access the mapping object in the object store;
• and
• generate a transaction log that tracks transactions of the object store.

A method of:

• hashing at least content of data objects to derive a data hash for each data object,
• wherein the data hashes are used to access the data objects, and
• wherein the data objects include one of:
• data, metadata, and data and metadata;
• generating a file object including a mapping of the data hashes for the data objects;
• hashing at least content of a file object to derive a file hash,
• wherein the file object is used to access the file object, and
• wherein the file hash is based on the data hashes;
• creating a mapping of an index node (inode) number to the file hash;
• storing the data objects in the object layer such that the data objects are accessed via the data hashes by the namespace layer;
• and
• storing the file object in the object layer such that the file object is accessed via the file hash by the namespace layer.