Back to results
US 11,960,441 B2Granted
Retention management for data streams
AssigneeEMC IP Holding Company LLC
InventorsMikhail Danilov, Igor Medvedev
Life of this patent
Estimating the term…
Abstract
The described technology is generally directed towards managing data retention policy for stream data stored in a streaming storage system. When a request to truncate a data stream from a certain position (e.g., from a request-specified stream cut) is received, an evaluation is made to determine whether the requested position is within a data retention period as specified by data retention policy. If any data prior to the stream cut position (corresponding to a stream cut time) is within the data retention period, the truncation request is blocked. Otherwise truncation from the stream cut point is allowed to proceed/is performed. Also described is handling automated (e.g., sized based) stream truncation requests with respect to data retention.
Claims20
A method, comprising:
- determining, by a streaming data storage system comprising a processor, whether a request to truncate data stream events in a data stream that are prior to a specified stream cut stored in an auxiliary stream associated with the data stream is allowable, wherein timing in the auxiliary stream and timing in the data stream are based on a same system timing protocol, and the determining comprising:
- obtaining a retention period associated with the data stream;
- obtaining a stream cut time associated with the specified stream cut;
- evaluating the stream cut time relative to the retention period to determine that the stream cut time is within the retention period or that the stream cut time is not within the retention period;
- in response to determining that the stream cut time is within the retention period;
- determining that the request to truncate the data stream events is not allowable, and
- blocking the request, wherein the request to truncate the data stream events is received from a size-based expiration requestor. and the blocking of the request corresponds to increasing a capacity footprint of the data stream beyond a pre-specified size limit;
- and
- in response to determining that the stream cut time is not within the retention period;
- determining that the request to truncate the data stream events is allowable, and
- truncating the data stream based on the stream cut time in response to the request, wherein truncating the data stream comprises deleting the data stream events from the streaming data storage system.
8 more elements
A system, comprising:
- a processor;
- and
- a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
- determining whether a request to truncate data stream events is allowable, the data stream events being represented in a data stream prior to a specified stream cut stored in an auxiliary stream associated with the data stream, wherein timing in the auxiliary stream and timing in the data stream are based on a same system timing protocol, and the determining comprising,
- obtaining a retention period associated with the data stream;
- obtaining a stream cut time associated with the specified stream cut;
- evaluating the stream cut time relative to the retention period to determine that the stream cut time is within the retention period or that the stream cut time is not within the retention period;
- in response to determining that the stream cut time is within the retention period;
- determining that the request to truncate the data stream events is not allowable, and
- blocking the request, wherein the request to truncate the data stream events is received from a size-based expiration requestor. and the blocking of the request corresponds to increasing a capacity footprint of the data stream beyond a pre-specified size limit;
- and
- in response to determining that the stream cut time is not within the retention period:
- determining that the request to truncate the data stream events is allowable, and
- truncating the data stream based on the stream cut time in response to the request, wherein truncating the data stream comprises deleting the data stream events from the streaming data storage system.
11 more elements
A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
- determining whether a request to truncate data stream events in a data stream, which are prior to a specified stream cut stored in an auxiliary stream associated with the data stream is allowable, wherein timing in the auxiliary stream and timing in the data stream are based on a same system timing protocol, and the determining comprising,
- obtaining a retention period associated with the data stream;
- obtaining a stream cut time associated with the specified stream cut;
- evaluating the stream cut time relative to the retention period to determine that the stream cut time is within the retention period or that the stream cut time is not within the retention period;
- in response to determining that the stream cut time is within the retention period;
- determining that the request to truncate the data stream events is not allowable, and
- blocking the request, wherein the request to truncate the data stream events is received from a size-based expiration requestor. and the blocking of the request corresponds to increasing a capacity footprint of the data stream beyond a pre-specified size limit;
- and
- in response to determining that the stream cut time is not within the retention period;
- determining that the request to truncate the data stream events is allowable, and
- truncating the data stream based on the stream cut time in response to the request, wherein truncating the data stream comprises deleting the data stream events from the streaming data storage system.
8 more elements
Dependencies are inferred from the claim text.