Parallel processing of filtered transaction logs
AssigneeWestern Digital Technologies, Inc.
InventorsThomas Demoor, Satish Kumar Chakka, Viswanath Chandrasekara Bharathi
Life of this patent
Estimating the term…
Abstract
Example storage systems and methods provide data storage management using parallel processing of filtered transaction logs. Transaction logs are comprised of log entries corresponding to storage operations for at least one storage node. Sets of log entries are sequentially retrieved from the transaction log and filtered through multiple transaction log filters to generate multiple subsets of the log entries. Different metadata operations are executed in parallel using the different filtered subsets of log entries.
Claims20
A storage system, comprising:
- at least one storage node configured to execute storage operations;
- a transaction log configured to store a plurality of log entries corresponding to storage operations for the at least one storage node;
- a first metadata streamer configured to sequentially retrieve a set of log entries from the transaction log;
- and
- a plurality of transaction log filters configured to selectively provide subsets of the set of log entries from the first metadata streamer to a plurality of metadata operations, wherein the metadata operations of the plurality of metadata operations are configured to operate in parallel on the subsets of the set of log entries.
A computer-implemented method, comprising:
- storing a transaction log comprised of a plurality of log entries corresponding to storage operations for at least one storage node;
- sequentially retrieving a set of log entries from the transaction log;
- selectively providing, using a first transaction log filter, a first subset of the set of log entries to a first metadata operation;
- selectively providing, using a second transaction log filter, a second subset of the set of log entries to a second metadata operation;
- and
- executing in parallel:
- the first metadata operation on the first subset of the set of log entries;
- and
- the second metadata operation on the second subset of the set of log entries.
A system, comprising:
- at least one storage node configured to execute storage operations;
- a transaction log comprised of a plurality of log entries corresponding to storage operations for the at least one storage node;
- means for sequentially retrieving a set of log entries from the transaction log;
- means for selectively providing, using a first transaction log filter, a first subset of the set of log entries to a first metadata operation;
- means for selectively providing, using a second transaction log filter, a second subset of the set of log entries to a second metadata operation;
- and
- means for executing in parallel:
- the first metadata operation on the first subset of the set of log entries;
- and
- the second metadata operation on the second subset of the set of log entries.
Dependencies are inferred from the claim text.