Back to results
US 10,951,465 B1Granted
Distributed file system analytics
AssigneeEMC IP Holding Company LLC
InventorsNishesh Gupta, Sandeep Nirmale, Deep Desai
Life of this patent
Estimating the term…
Abstract
A method, article of manufacture, and apparatus for analyzing a distributed file system is discussed. A metadata operation is captured in an audit log, wherein the metadata operation is on an object stored in the distributed file system. An event is pushed to a distributed commit log and a real-time analytics engine in response to capturing the metadata operation. The event is persistently stored in the distributed commit log.
Claims20
A method comprising:
- intercepting a request transmitted by a client to a metadata server (MDS), and the request transmitted by the client implicates a metadata operation;
- capturing the metadata operation in an audit log of the metadata server (MDS), wherein the metadata operation is on an object stored in a distributed file system (DFS);
- pushing, by the metadata server (MDS), an event to a distributed commit log and to a real-time analytics engine that is in communication with a plurality of audit logs including the audit log of the metadata server (MDS), and the real-time analytics engine collects analytics information about the DFS from the audit log and cooperates with the distributed commit log to enable client access to metadata spread across a plurality of metadata servers (MDS), and the real-time analytics engine is smaller than the distributed commit log and the real-time analytics engine stores only live data and not legacy information, and the event is pushed to the distributed commit log and the real-time analytics engine in response to capturing the metadata operation, wherein the distributed commit log is shared by the plurality of metadata servers (MDS) which are each associated with a respective audit log and the distributed commit log receives events from the plurality of metadata servers (MDS) that have been captured in the respective audit logs of those metadata servers (MDS);
- and
- persistently storing the event in the distributed commit log.
2 more elements
A non-transitory storage medium having stored therein instructions which are executable by one or more processors to perform operations comprising:
- intercepting a request transmitted by a client to a metadata server (MDS), and the request transmitted by the client implicates a metadata operation;
- capturing the metadata operation in an audit log of the metadata server (MDS), wherein the metadata operation is on an object stored in a distributed file system (DFS);
- pushing, by the metadata server (MDS), an event to a distributed commit log and to a real-time analytics engine that is in communication with a plurality of audit logs including the audit log of the metadata server (MDS), and the real-time analytics engine collects analytics information about the DFS from the audit log and cooperates with the distributed commit log to enable client access to metadata spread across a plurality of metadata servers (MDS), and the real-time analytics engine is smaller than the distributed commit log and the real-time analytics engine stores only live data and not legacy information, and the event is pushed to the distributed commit log and the real-time analytics engine in response to capturing the metadata operation, wherein the distributed commit log is shared by the plurality of metadata servers (MDS) which are each associated with a respective audit log and the distributed commit log receives events from the plurality of metadata servers (MDS) that have been captured in the respective audit logs of those metadata servers (MDS);
- and
- persistently storing the event in the distributed commit log.
2 more elements
A system comprising a computer processor configured to execute instructions for performing operations comprising:
- intercepting a request transmitted by a client to a metadata server (MDS), and the request transmitted by the client implicates a metadata operation;
- capturing the metadata operation in an audit log of the metadata server (MDS), wherein the metadata operation is on an object stored in a distributed file system (DFS);
- pushing, by the metadata server (MDS), an event to a distributed commit log and to a real-time analytics engine that is in communication with a plurality of audit logs including the audit log of the metadata server (MDS), and the real-time analytics engine collects analytics information about the DFS from the audit log and cooperates with the distributed commit log to enable client access to metadata spread across a plurality of metadata servers (MDS), and the real-time analytics engine is smaller than the distributed commit log and the real-time analytics engine stores only live data and not legacy information, and the event is pushed to the distributed commit log and the real-time analytics engine in response to capturing the metadata operation, wherein the distributed commit log is shared by the plurality of metadata servers (MDS) which are each associated with a respective audit log and the distributed commit log receives events from the plurality of metadata servers (MDS) that have been captured in the respective audit logs of those metadata servers (MDS);
- and
- persistently storing the event in the distributed commit log.
2 more elements
Dependencies are inferred from the claim text.