US 11,144,504 B1Granted

Eliminating redundant file system operations

AssigneeEMC IP Holding Company LLC

InventorsDeepti Chheda, Diwaker Gupta, Vaibhav Kamra, Nathan Rosenblum, Niraj Tolia

Life of this patent

Estimating the term…

Abstract

Techniques to eliminate redundant file system operations are disclosed. In various embodiments, an indication is received to send locally-stored file system operations to a remote file system metadata server. A set of related locally-stored file system operations is determined. Operations in the set are determined to satisfy a merger criteria and are merged into a single merged file system operation. The merged file system operation is sent to the remote file system metadata server.

Claims24

Dependent claims

A method of processing file system operations, comprising:

writing, by one or more processors of a client connected to a distributed file system including a remote file system metadata server, one or more file system operations affecting metadata for which a lease is held to a buffer at the client, the lease being held with respect to at least a portion of a file system metadata;
obtaining, by the client, an indication that the lease is to end;
in response to obtaining the indication that the lease is to end, determining, by the client, to send locally-stored file system operations to the remote file system metadata server, the locally-stored file system operations comprising at least one of the one or more file system operations written to the buffer;
and
in response to determining to send locally-stored file system operations to the remote file system metadata server,
determining, by the client, a set of related locally-stored file system operations to be communicated to the file system metadata server,
wherein:
the determining the set of related locally-stored file system operations comprises identifying from among a plurality of file system operations a linear chain of dependent file system operations based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations,
the identifying the linear chain of dependent file system operations processing the dependency graph to find the linear chain,
the identifying the linear chain being based at least in part on a determination that a sequence of operations each have at most a single dependency on a prior operation in the linear chain and none of which has more than one inbound dependency;
identifying, by the client, at least a subset of operations in the set as satisfying a merger criteria;
merging, by the client, the subset of operations into a single merged file system operation;
and
sending, by the client, the merged file system operation to the remote file system metadata server, wherein the merged file system operation is sent based at least in part on the receiving of the indication, wherein sending the merged file system operation includes ensuring that an operation of the one or more file system operations is not sent to the remote file system metadata server unless all operations on which the operation depends have already been sent to the remote file system metadata server or are being sent in connection with the operation.

11 more elements

A system, comprising:

a communication interface;
and
a processor of a client connected to a distributed file system including a remote file system metadata server, the processor coupled to the communication interface and configured to:
write one or more file system operations affecting metadata for which a lease is held to a buffer at the client, the lease being held with respect to at least a portion of a file system metadata;
obtain an indication that the lease is to end
in response to obtaining the indication that the lease is to end, determine to send locally-stored file system operations to the remote file system metadata server, the locally-stored file system operations comprising at least one of the one or more file system operations written to the buffer;
and
in response to determining to send locally-stored file system operations to the remote file system metadata server,
determine a set of related locally-stored file system operations to be communicated to the file system metadata server,
wherein:
to determine the set of related locally-stored file system operations comprises identifying from among a plurality of file system operations a linear chain of dependent file system operations based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations,
the identifying the linear chain of dependent file system operations processing the dependency graph to find the linear chain,
the linear chain being based at least in part on a determination that a sequence of operations each have at most a single dependency on a prior operation in the linear chain and none of which has more than one inbound dependency;
identify at least a subset of operations in the set as satisfying a merger criteria;
merge the subset of operations into a single merged file system operation;
and
send the merged file system operation to the remote file system metadata server via the communication interface, wherein the merged file system operation is sent based at least in part on the receiving of the indication, wherein sending the merged file system operation includes ensuring that an operation of the one or more file system operations is not sent to the remote file system metadata server unless all operations on which the operation depends have already been sent to the remote file system metadata server or are being sent in connection with the operation.

14 more elements

A computer program product to process file system operations, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

writing, by one or more processors of a client connected to a distributed file system including a remote file system metadata server, one or more file system operations affecting metadata for which a lease is held to a buffer at the client, the lease being held with respect to at least a portion of a file system metadata;
obtaining, by the client, an indication that the lease is to end;
in response to obtaining the indication that the lease is to end, determining, by the client, to send locally-stored file system operations to the remote file system metadata server, the locally-stored file system operations comprising at least one of the one or more file system operations written to the buffer;
and
in response to determining to send locally-stored file system operations to the remote file system metadata server,
determining, by the client, a set of related locally-stored file system operations to be communicated to the file system metadata server,
wherein:
the determining the set of related locally-stored file system operations comprises identifying from among a plurality of file system operations a linear chain of dependent file system operations based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations,
the identifying the linear chain of dependent file system operations processing the dependency graph to find the linear chain,
the identifying the linear chain being based at least in part on a determination that a sequence of operations each have at most a single dependency on a prior operation in the linear chain and none of which has more than one inbound dependency;
identifying, by the client, at least a subset of operations in the set as satisfying a merger criteria;
merging, by the client, the subset of operations into a single merged file system operation;
and
sending, by the client, the merged file system operation to the remote file system metadata server, wherein the merged file system operation is sent based at least in part on the receiving of the indication, wherein sending the merged file system operation includes ensuring that an operation of the one or more file system operations is not sent to the remote file system metadata server unless all operations on which the operation depends have already been sent to the remote file system metadata server or are being sent in connection with the operation.

11 more elements

Dependencies are inferred from the claim text.

patentgrep

Loading patent…

Abstract

A method of processing file system operations, comprising:

writing, by one or more processors of a client connected to a distributed file system including a remote file system metadata server, one or more file system operations affecting metadata for which a lease is held to a buffer at the client, the lease being held with respect to at least a portion of a file system metadata;
obtaining, by the client, an indication that the lease is to end;
in response to obtaining the indication that the lease is to end, determining, by the client, to send locally-stored file system operations to the remote file system metadata server, the locally-stored file system operations comprising at least one of the one or more file system operations written to the buffer;
and
in response to determining to send locally-stored file system operations to the remote file system metadata server,
determining, by the client, a set of related locally-stored file system operations to be communicated to the file system metadata server,
wherein:
the determining the set of related locally-stored file system operations comprises identifying from among a plurality of file system operations a linear chain of dependent file system operations based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations,
the identifying the linear chain of dependent file system operations processing the dependency graph to find the linear chain,
the identifying the linear chain being based at least in part on a determination that a sequence of operations each have at most a single dependency on a prior operation in the linear chain and none of which has more than one inbound dependency;
identifying, by the client, at least a subset of operations in the set as satisfying a merger criteria;
merging, by the client, the subset of operations into a single merged file system operation;
and
sending, by the client, the merged file system operation to the remote file system metadata server, wherein the merged file system operation is sent based at least in part on the receiving of the indication, wherein sending the merged file system operation includes ensuring that an operation of the one or more file system operations is not sent to the remote file system metadata server unless all operations on which the operation depends have already been sent to the remote file system metadata server or are being sent in connection with the operation.

11 more elements