US 8,554,809 B1Granted

Calculating quota usage without impacting file system services

AssigneeEMC Corporation

InventorsYingchao Zhou, Wei Kang, Weigang Zhong, Chen Gong, Kevin Xu, Sitaram Pawar

Life of this patent

Estimating the term…

Loading patent…

Abstract

An online tree quota check tool computes quota usage of files in a directory tree while the structure of the directory tree keeps changing due to concurrent write access by other applications and clients. The quota check tool computes the quota usage without interfering with the ongoing file access operations that change the directory content and quota usage. A quota check database keeps track of which files have been checked and which have not, so that each file is checked and its quota usage is accumulated once and only once while directories are moved and directory scans are restarted as a result of the changing directory content. File system operations are modified to synchronize with the online tree quota check. The quota check tool can be single threaded or multi-threaded. A multi-threaded tool dispatches idle directory iteration threads to scan subdirectories in the directory tree.

Claims20

Dependent claims

A method comprising a data processor of a file server executing computer instructions stored on a non-transitory computer-readable storage medium to perform the steps of:

(a) accumulating quota usage of files in a directory tree when scanning the directory tree and a network client has concurrent write access to the files in the directory tree and changes directory structure of the directory tree;
(b) marking each file in the directory tree as checked for quota usage when the quota usage of said each file is accumulated, and inspecting the files for checked status during the scanning of the directory tree so that quota usage of checked files is not accumulated more than once during the scanning of the directory tree;
(c) in response to a rename request by the network client to rename a specified file concurrent with the scanning of the directory tree when the rename request specifies a destination directory that is found to be marked as checked, accumulating quota usage of the specified file;
(d) restarting a scan of a modified directory in the directory tree to accumulate quota usage of files in the modified directory when the concurrent write access of the network client has modified content of the modified directory during the scan;
and
(e) using the accumulated quota usage from the steps (a), (c), and (d) to set quota running counts maintained by a file system manager in the file server and used by the file system manager to deny a network client request that would cause a quota running count to exceed a quota limit.

3 more elements

A file server comprising:

data storage storing a file system;
a data processor coupled to the data storage for accessing the file system;
a network adapter for connecting network clients to the data processor for receiving file access requests from the network clients and exchanging file data between the network clients and the data processor;
and
non-transitory computer readable storage medium coupled to the data processor and storing computer instructions that when executed by the data processor perform the steps of:
(a) accumulating quota usage of files in a directory tree of the file system when scanning the directory tree and a network client has concurrent write access to the files in the directory tree and changes directory structure of the directory tree;
(b) marking each file in the directory tree as checked for quota usage when the quota usage of said each file is accumulated, and inspecting the files for checked status during the scanning of the directory tree so that quota usage of checked files is not accumulated more than once during the scanning of the directory tree;
(c) in response to a rename request by the network client to rename a specified file concurrent with the scanning of the directory tree when the rename request specifies a destination directory that is found to be marked as checked, accumulating quota usage of the specified file;
and
(d) restarting a scan of a modified directory in the directory tree to accumulate quota usage of files in the modified directory when the concurrent write access of the network client has modified content of the modified directory during the scan.

7 more elements

A file server comprising:

data storage storing a file system;
a data processor coupled to the data storage for accessing the file system;
a network adapter for connecting network clients to the data processor for receiving file access requests from the network clients and exchanging file data between the network clients and the data processor;
and
non-transitory computer readable storage medium coupled to the data processor and storing computer instructions that when executed by the data processor perform the steps of:
(a) accumulating quota usage of files in a directory tree of the file system when scanning the directory tree and a network client has concurrent write access to the files in the directory tree and changes directory structure of the directory tree;
(b) marking each file in the directory tree as checked for quota usage when the quota usage of said each file is accumulated, and inspecting the files for checked status during the scanning of the directory tree so that quota usage of checked files is not accumulated more than once during the scanning of the directory tree;
(c) in response to a rename request by the network client to rename a specified file concurrent with the scanning of the directory tree when the rename request specifies a destination directory that is found to be marked as checked, accumulating quota usage of the specified file;
and
(d) restarting a scan of a modified directory in the directory tree to accumulate quota usage of files in the modified directory when the concurrent write access of the network client has modified content of the modified directory during the scan;
wherein step (d) includes inspecting a modification time attribute of the modified directory, and restarting the scan of the modified directory in response to finding that the modification time attribute of the modified directory has changed while scanning the modified directory to accumulate quota usage of the files in the modified directory.

8 more elements

A file server comprising:

data storage storing a file system;
a data processor coupled to the data storage for accessing the file system;
a network adapter for connecting network clients to the data processor for receiving file access requests from the network clients and exchanging file data between the network clients and the data processor;
and
non-transitory computer readable storage medium coupled to the data processor and storing computer instructions;
wherein the computer instructions include:
a quota check management component for initiating an online tree quota check of a directory tree of the file system while a network client has concurrent write access to the files in the directory tree and changes directory structure of the directory tree;
a quota check directory iteration routine for scanning a specified directory of the directory tree and accumulating quota usage of files in the specified directory and initiating scans of subdirectories of the specified directory to accumulate quota usage of files in the subdirectories;
a quota check database access routine for accessing a quota check database of files that have been checked for quota usage since an initiation of said online tree quota check;
and
modified file access routines for use by a file system manager during the online quota check;
wherein the computer instructions, when executed by the data processor, perform the steps of:
(a) performing the online tree quota check of the directory tree of the file system by executing the quota check directory iteration routine to scan directories of the directory tree while the network client has concurrent write access to the files in the directory tree and changes directory structure of the directory tree;
(b) accessing the quota check database to keep a record of files in the directory tree that have been checked for quota usage by accumulating quota usage, and accessing the quota check database during the scanning of the directory tree so that quota usage of checked files is not accumulated more than once during the scanning of the directory tree;
(c) in response to a rename request by the network client to rename a specified file concurrent with the scanning of the directory tree when the rename request specifies a destination directory that is found in the quota check database, accumulating quota usage of the specified file;
and
(d) restarting a scan of a modified directory in the directory tree to accumulate quota usage of files in the modified directory when the concurrent write access of the network client has modified content of the modified directory during the scan.

14 more elements

A file server comprising:

data storage storing a file system;
a data processor coupled to the data storage for accessing the file system;
a network adapter for connecting network clients to the data processor for receiving file access requests from the network clients and exchanging file data between the network clients and the data processor;
and
non-transitory computer readable storage medium coupled to the data processor and storing computer instructions;
wherein the computer instructions include:
a quota check management component for initiating an online tree quota check of a directory tree of the file system while a network client has concurrent write access to the files in the directory tree and changes directory structure of the directory tree;
a quota check directory iteration routine for scanning a specified directory of the directory tree and accumulating quota usage of files in the specified directory and initiating scans of subdirectories of the specified directory to accumulate quota usage of files in the subdirectories;
a quota check database access routine for accessing a quota check database of files that have been checked for quota usage since an initiation of said online tree quota check;
and
modified file access routines for use by a file system manager during the online quota check;
wherein the computer instructions, when executed by the data processor, perform the steps of:
(a) performing the online tree quota check of the directory tree of the file system by executing the quota check directory iteration routine to scan directories of the directory tree while the network client has concurrent write access to the files in the directory tree and changes directory structure of the directory tree;
(b) accessing the quota check database to keep a record of files in the directory tree that have been checked for quota usage by accumulating quota usage, and accessing the quota check database during the scanning of the directory tree so that quota usage of checked files is not accumulated more than once during the scanning of the directory tree;
(c) in response to a rename request by the network client to rename a specified file concurrent with the scanning of the directory tree when the rename request specifies a destination directory that is found in the quota check database, accumulating quota usage of the specified file;
and
(d) restarting a scan of a modified directory in the directory tree to accumulate quota usage of files in the modified directory when the concurrent write access of the network client has modified content of the modified directory during the scan;
wherein the directory iteration routine, when executed by the data processor, checks whether there is an idle directory iteration thread in a pool of directory iteration threads, and upon finding that there is not an idle directory iteration thread in the pool of directory iteration threads, the directory iteration routine scans the subdirectory to accumulate quota usage of files in the subdirectory;
and
wherein the directory iteration routine, when executed by the data processor, saves an old value of a modification time attribute of the specified directory prior to scanning the subdirectory to accumulate quota usage of files in the subdirectory, and upon completing the scanning the subdirectory to accumulate quota usage of files in the subdirectory, compares the old value of the modification time attribute of the specified directory to a new value of the modification time attribute of the specified directory to determine that the specified directory has been modified, and upon determining that the specified directory has been modified, restarting a scan of the specified directory to accumulate quota usage of files in the specified directory.

17 more elements

Dependencies are inferred from the claim text.