# US 2021/0342296 A1 · Retention management for data streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342296 A1
- Document ID: US20210342296A1
- Status: Published application
- Kind: A1
- Application number: 16/864,892
- Filed: 2020-05-01
- Published: 2021-11-04
- Earliest claimed: 2020-05-01
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Mikhail Danilov; Igor Medvedev
- Web page: https://patentgrep.com/patent/US20210342296A1

## CPC

- G06F 16/125: {characterised by the use of retention policies (retention policies for HSM systems G06F16/185)}
- G06F 16/1734: {Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs}
- G06F 16/182: Distributed file systems
- G06F 16/24568: {Data stream processing; Continuous queries}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The described technology is generally directed towards managing data retention policy for stream data stored in a streaming storage system. When a request to truncate a data stream from a certain position (e.g., from a request-specified stream cut) is received, an evaluation is made to determine whether the requested position is within a data retention period as specified by data retention policy. If any data prior to the stream cut position (corresponding to a stream cut time) is within the data retention period, the truncation request is blocked. Otherwise truncation from the stream cut point is allowed to proceed/is performed. Also described is handling automated (e.g., sized based) stream truncation requests with respect to data retention.

## Claims

### Claim 1

A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
receiving a request to truncate a data stream of a streaming data storage system, the request associated with a stream cut maintained for the data stream;
obtaining a retention period associated with the data stream;
obtaining a stream cut time associated with the stream cut;
determining whether the stream cut time is within the retention period; and
in response to the stream cut time being determined not to be within the retention period, truncating the data stream based on the stream cut.

### Claim 2 (depends on 1)

The system of claim 1, wherein the determining whether the stream cut time is within the retention period comprises determining a retention end time based on the stream cut time and the retention period, and determining whether the retention end time is later than a current system time.

### Claim 3 (depends on 1)

The system of claim 1, wherein the operations further comprise, in response to the stream cut time being determined to be within the retention period, blocking the request.

### Claim 4 (depends on 3)

The system of claim 3, wherein the request to truncate the data stream is received from a size-based expiration requestor, and wherein the blocking the request corresponds to increasing a capacity footprint of the data stream beyond a pre-specified size limit.

### Claim 5 (depends on 4)

The system of claim 4, wherein the operations further comprise scheduling a delayed truncation operation in response to the increasing the capacity footprint of the data stream.

### Claim 6 (depends on 5)

The system of claim 5, wherein the scheduling the delayed truncation operation comprises scheduling the delayed truncation operation to occur at a stream cut time that is not within the retention time period.

### Claim 7 (depends on 1)

The system of claim 1, wherein the stream cut is a second stream cut and the stream cut time value is a second stream cut time value that is after a first stream cut time value of a first stream cut earlier in time than the second stream cut, and wherein the operations further comprise,
in response to determining that the second stream cut time value is within the retention period, selecting the first stream cut, and determining whether the first stream cut time value is within the retention period, and
in response to determining that the first stream cut time is not within the retention period, truncating the data stream based on the first stream cut.

### Claim 8 (depends on 1)

The system of claim 1, wherein the operations further comprise maintaining an auxiliary stream comprising stream cut data for the stream cut in association with the data stream.

### Claim 9 (depends on 8)

The system of claim 8, wherein the operations further comprise, in response to the stream cut time being determined not to be within the retention period, truncating the auxiliary stream based on the stream cut.

### Claim 10

A method comprising:
determining, by a streaming data storage system comprising a processor, whether a request to truncate data stream events in a data stream that are prior to a specified stream cut is allowable, the determining comprising,
obtaining a retention period associated with the data stream;
obtaining a stream cut time associated with the specified stream cut; and
evaluating the stream cut time relative to the retention period to determine whether the stream cut time is within the retention period or whether the stream cut time is not within the retention period;
in response to determining that the stream cut time is within the retention period, determining that the request to truncate the data stream events is not allowable; and
in response to determining that the stream cut time is not within the retention period, determining that the request to truncate the data stream events is allowable, and truncating the data stream based on the stream cut in response to the request.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
in response to determining that the request to truncate the data stream events is not allowable, blocking the request.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
in response to determining that the request to truncate the data stream events is not allowable, scheduling a future truncation operation based on the specified stream cut.

### Claim 13 (depends on 10)

The method of claim 10, wherein the evaluating the stream cut time relative to the retention period to determine whether the stream cut time is within the retention period or whether the stream cut time is not within the retention period comprises determining a retention period end time based on the stream cut time and the retention period, and determining whether the retention end time is later than a current system time.

### Claim 14 (depends on 10)

The method of claim 10, wherein the evaluating the stream cut time relative to the retention period to determine whether the stream cut time is within the retention period or whether the stream cut time is not within the retention period comprises determining a retention period start time based on a current system time and the retention period, and determining whether the stream cut time is earlier than the retention period start time.

### Claim 15 (depends on 10)

The method of claim 10, further comprising:
maintaining an auxiliary stream comprising stream cut data for the stream cut in association with the data stream, and, in response to determining that the stream cut time has a stream cut time value that is not within the retention period, truncating the auxiliary stream based on the stream cut.

### Claim 16

A machine-readable storage medium, comprising executable instructions that, when executed by a processor of a streaming data storage system, facilitate performance of operations, the operations comprising:
receiving a request to truncate events of a data stream of the streaming data storage system, in which the events are relative to a specified stream cut;
determining whether the events are subject to a retention policy based on a retention period associated with the data stream; and
in response to determining that the events are not subject to the retention policy, truncating the events based on the specified stream cut.

### Claim 17 (depends on 16)

The machine-readable storage medium of claim 16, wherein the determining whether the events are subject to the retention policy comprises accessing a stream cut time value associated with the specified stream cut, and determining whether the stream cut time value is within the retention period.

### Claim 18 (depends on 16)

The machine-readable storage medium of claim 16, wherein the operations further comprise, in response to determining that the events are subject to the retention policy, blocking the request.

### Claim 19 (depends on 16)

The machine-readable storage medium of claim 16, wherein the operations further comprise, in response to determining that the events are subject to the retention policy, scheduling a future truncation operation based on the specified stream cut.

### Claim 20 (depends on 16)

The machine-readable storage medium of claim 16, wherein the operations further comprise, in response to determining that the events are subject to the retention policy, attempting to locate an earlier stream cut that is prior to the specified stream cut.
