Organizing present and future reads from a tiered streaming data storage layer
AssigneeEMC IP Holding Company LLC
InventorAndrei Paduroiu
Life of this patent
Estimating the term…
Abstract
Implementations are provided herein for organizing present and future reads from a tiered streaming data storage layer. Implementations allow for access to multi-tiered streaming data organized in different append-only segments, some of which may be related to each other. Streaming data can be read from fast local tier 1 storage, streaming data can be retrieved from fold tier 2 storage, and registrations can be made to read streaming data that has not yet been written to the storage layer.
Claims18
A method comprising:
- receiving a lookup request from a caller, wherein the lookup request includes a segment, an offset, and a maximum length, wherein the offset identifies a starting position of a block of data corresponding to the segment and the maximum length corresponds to a number of bytes;
- and
- establishing an iterator based on the segment and the offset, wherein the iterator returns one of the following to the caller for each iteration of a period based on the maximum length:
- a next set of data responsive to the lookup request wherein the next set of data is stored in a cache memory of a receiving device of the lookup request;
- a storage read result;
- or
- a future read result.
A system comprising at least one storage device and at least one hardware processor configured to:
- receive a lookup request from a caller, wherein the lookup request includes a segment, an offset, and a maximum length, wherein the offset identifies a starting position of a block of data corresponding to the segment and the maximum length corresponds to a number of bytes;
- and
- establish an iterator based on the segment and the offset, wherein the iterator returns one of the following to the caller for each iteration of a period based on the maximum length:
- a next set of data responsive to the lookup request wherein the next set of data is stored in a cache memory of a receiving device of the lookup request;
- a storage read result;
- or
- a future read result.
A non-transitory computer readable medium with program instructions stored thereon to perform the following acts:
- receiving a lookup request from a caller, wherein the lookup request includes a segment, an offset, and a maximum length, wherein the offset identifies a starting position of a block of data corresponding to the segment and the maximum length corresponds to a number of bytes;
- and
- establishing an iterator based on the segment and the offset, wherein the iterator returns one of the following to the caller for each iteration of a period based on the maximum length:
- a next set of data responsive to the lookup request wherein the next set of data is stored in a cache memory of a receiving device of the lookup request;
- a storage read result;
- or
- a future read result.
Dependencies are inferred from the claim text.