Stream retention in a data storage system

#The subject application is related to data storage, and more particularly, to techniques for managing data retention in a data storage system.

#As advancements in computing technology have increased the amount and scope of data that can be processed by a computing system, data storage systems have similarly seen significant advances to keep pace with the technical landscape. For instance, in addition to traditional storage systems that work on the basis of blocks, files, objects, and/or other similar data structures, stream-based data storage systems have been developed that implement data sequences called streams as a mechanism to store and/or serve continuous and potentially unbounded data.

#Whereas a data stream as managed by a data storage system can be potentially unbounded, storage resources are normally limited. As a result, a stream-based data storage system can facilitate data expiration, data truncation, and/or other means to purge recorded data after a given length of time to allocate storage resources for new data.

#Various non-limiting embodiments of the subject disclosure are described with reference to the following figures, wherein like reference numerals refer to like parts throughout unless otherwise specified.

#FIG. 1 is a block diagram of a system that facilitates stream retention in a data storage system in accordance with various aspects described herein.

#FIG. 2 is a diagram depicting an example synthetic stream structure that can be utilized in accordance with various aspects described herein.

#Various specific details of the disclosed embodiments are provided in the description below. One skilled in the art will recognize, however, that the techniques described herein can in some cases be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

#Unlike conventional data storage systems that work with structures such as blocks, files, and objects, stream storage systems provide a new storage abstraction, referred to as a stream, for continuous and potentially unbounded data. As used herein, the term “stream” refers to a durable, elastic, append-only, unbounded sequence of data. Stated another way, a stream storage system implements streams as a first class primitive for storing and/or serving continuous and unbounded data.

#In addition, data associated with a stream can be grouped within the stream into constructs referred to as events. By way of specific, non-limiting example, a blockchain ledger can be stored as a stream, where respective events in the stream correspond to individual contracts. As another example, data collected over time from sensors and/or other devices can be stored and/or maintained as a stream. In general, however, a stream can be utilized for the storage of any suitable unbounded data sequences.