# US 10,481,980 B2 · Enabling segmented source data introspection within dispersed storage network (DSN) memory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,980 B2
- Document ID: US10481980B2
- Status: Granted
- Kind: B2
- Application number: 15/940,396
- Filed: 2018-03-29
- Published: 2019-10-03
- Granted: 2019-11-19
- Earliest claimed: 2018-03-29
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Bryan J. Keller; Brian F. Ober
- Web page: https://patentgrep.com/patent/US10481980B2

## CPC

- G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems}
- G06F 3/0604: {Improving or facilitating administration, e.g. storage management}
- G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors}
- G06F 3/064: {Management of blocks}
- G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)}
- H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
- G06F 2211/1028: Distributed, i.e. distributed RAID systems with parity
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A method begins by a first computing device obtaining data for storage in a dispersed storage network (DSN) and determining whether to divide the data into a plurality of independent data units (IDUs). When determined to divide the data, the method continues by dividing the data into the plurality of IDUs, generating an aggregation indicator that links the plurality of IDUs to the data and sending DSN write processing requests and the aggregation indicator to DS processing units. The method continues with a first dispersed storage (DS) processing unit receiving a first DSN write processing request and the aggregation indicator. The method continues by the first DS processing unit dispersed storage error encoding the first IDU to produce encoded data slices, generating aggregation information regarding the plurality of IDUs based on the aggregation indicator and sending the encoded data slices to storage units of the DSN for storage therein.

## Claims

### Claim 1

A method comprises:
obtaining, by a first computing device, data for storage in a dispersed storage network (DSN);
determining, by the first computing device, whether to divide the data into a plurality of independent data units;
when determined to divide the data:
dividing, by the first computing device, the data into the plurality of independent data units;
generating, by the first computing device, an aggregation indicator that links the plurality of independent data units to the data;
sending, by the first computing device, DSN write processing requests and the aggregation indicator to DS processing units of the DSN regarding at least some of the plurality of independent data units;
receiving, by a first dispersed storage (DS) processing unit of the DS processing units, a first DSN write processing request of the DSN write processing requests and the aggregation indicator, wherein the first DSN write processing request is regarding a first independent data unit of the plurality of independent data units;
dispersed storage error encoding, by the first DS processing unit, the first independent data unit to produce a first plurality of sets of encoded data slices;
generating, by the first DS processing unit, aggregation information regarding the plurality of independent data units based on the aggregation indicator, wherein the first DS processing unit utilizes the aggregation information to process a retrieval request regarding the data or the first independent data unit;
sending, by the first DS processing unit, the first plurality of sets of encoded data slices to a first set of storage units of the DSN for storage therein;
determining, by the first DS processing unit, whether the first plurality of sets of encoded data slices have been successfully stored in the first set of storage units;
when the first plurality of sets of encoded data slices have not been successfully stored, generating, by the first DS processing unit, an indication to reflect that the first independent data unit is a non-addressable data unit; and
when the first plurality of sets of encoded data slices have been successfully stored, generating, by the first DS processing unit, the indication to reflect that the first independent data unit is an addressable unit.

### Claim 2 (depends on 1)

The method of claim 1, wherein the indication is made public to the DSN.

### Claim 3 (depends on 1)

The method of claim 1 further comprises:
when not determined to divide the data:
dispersed storage error encoding, by the first computing device, the data to produce one or more sets of encoded data slices; and
sending, by the first computing device, at least one of the one or more sets of encoded data slices to storage units of the DSN for storage therein.

### Claim 4 (depends on 1)

The method of claim 1 further comprises:
receiving, by a second DS processing unit of the DS processing units, a second DSN write processing request of the DSN write processing requests and the aggregation indicator, wherein the second DSN write processing request is regarding a second independent data unit of the plurality of independent data units;
dispersed storage error encoding, by the second DS processing unit, the first independent data unit to produce a second plurality of sets of encoded data slices;
generating, by the second DS processing unit, second aggregation information regarding the plurality of independent data units based on the aggregation indicator, wherein the second DS processing unit utilizes the second aggregation information to process a retrieval request regarding the data or the second independent data unit;
sending, by the second DS processing unit, the second plurality of sets of encoded data slices to a second set of storage units of the DSN for storage therein;
determining, by the second DS processing unit, whether the second plurality of sets of encoded data slices have been successfully stored in the second set of storage units;
when the second plurality of sets of encoded data slices have not been successfully stored, generating, by the second DS processing unit, a second indication to reflect that the second independent data unit is a non-addressable data unit; and
when the second plurality of sets of encoded data slices have been successfully stored, generating, by the second DS processing unit, the second indication to reflect that the second independent data unit is an addressable unit.

### Claim 5 (depends on 1)

The method of claim 1, wherein the aggregation indicator comprises one or more of:
a field within the first DSN write processing request that indicates whether the data is aggregated or non-aggregated;
textual mapping data regarding how the data is aggregated; and
information regarding the plurality of independent data units.

### Claim 6 (depends on 1)

The method of claim 1, wherein the aggregation information comprises one of:
a copy of the aggregation indicator; and
a portion of the aggregation indicator.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first computing device is the first DS processing unit.

### Claim 8

A non-transitory computer readable storage medium comprises:
a first memory element that stores operational instructions that, when executed by a first computing device of a dispersed storage network (DSN), causes the first computing device to:
obtain data for storage in a dispersed storage network (DSN);
a second memory element that stores operational instructions that, when executed by the first computing device, causes the first computing device to:
determine whether to divide the data into a plurality of independent data units; and
when determined to divide the data:
divide the data into the plurality of independent data units;
generate an aggregation indicator that links the plurality of independent data units to the data; and
send DSN write processing requests and the aggregation indicator to DS processing units of the DSN regarding at least some of the plurality of independent data units;
a third memory element that stores operational instructions that, when executed by a first dispersed storage (DS) processing unit of the DS processing units, causes the first DS processing unit to:
receive a first DSN write processing request of the DSN write processing requests and the aggregation indicator, wherein the first DSN write processing request is regarding a first independent data unit of the plurality of independent data units;
dispersed storage error encode the first independent data unit to produce a first plurality of sets of encoded data slices;
generate aggregation information regarding the plurality of independent data units based on the aggregation indicator, wherein the first DS processing unit utilizes the aggregation information to process a retrieval request regarding the data or the first independent data unit;
send the first plurality of sets of encoded data slices to a first set of storage units of the DSN for storage therein;
a fourth memory element that stores operational instructions that, when executed by the first DS processing unit, causes the first DS processing unit to:
determine whether the first plurality of sets of encoded data slices have been successfully stored in the first set of storage units;
when the first plurality of sets of encoded data slices have not been successfully stored, generate an indication to reflect that the first independent data unit is a non-addressable data unit; and
when the first plurality of sets of encoded data slices have been successfully stored, generate the indication to reflect that the first independent data unit is an addressable unit.

### Claim 9 (depends on 8)

The non-transitory computer readable storage medium of claim 8, wherein the fourth memory element stores further operational instructions that, when executed by the first DS processing unit, cause the first DS processing unit to make the indication public to the DSN.

### Claim 10 (depends on 8)

The non-transitory computer readable storage medium of claim 8, wherein the second memory element stores further operational instructions that when executed by the first computing device, cause the first computing device to:
when not determined to divide the data:
dispersed storage error encode the data to produce one or more sets of encoded data slices; and
send at least one of the one or more sets of encoded data slices to storage units of the DSN for storage therein.

### Claim 11 (depends on 8)

The non-transitory computer readable storage medium of claim 8 further comprises:
a fifth memory element that stores operational instructions that, when executed by a second DS processing unit of the DSN, causes the second DS processing unit to:
receive a first DSN write processing request of the DSN write processing requests and the aggregation indicator, wherein the first DSN write processing request is regarding a first independent data unit of the plurality of independent data units;
dispersed storage error encode the first independent data unit to produce a first plurality of sets of encoded data slices;
generate aggregation information regarding the plurality of independent data units based on the aggregation indicator, wherein the first DS processing unit utilizes the aggregation information to process a retrieval request regarding the data or the first independent data unit;
send the first plurality of sets of encoded data slices to a first set of storage units of the DSN for storage therein; and
a sixth memory element that stores operational instructions that, when executed by the second DS processing unit, causes the second DS processing unit to:
determine whether the first plurality of sets of encoded data slices have been successfully stored in the first set of storage units;
when the first plurality of sets of encoded data slices have not been successfully stored, generate an indication to reflect that the first independent data unit is a non-addressable data unit; and
when the first plurality of sets of encoded data slices have been successfully stored, generate the indication to reflect that the first independent data unit is an addressable unit.

### Claim 12 (depends on 8)

The non-transitory computer readable storage medium of claim 8, wherein the aggregation indicator comprises one or more of:
a field within the first DSN write processing request that indicates whether the data is aggregated or non-aggregated;
textual mapping data regarding how the data is aggregated; and
information regarding the plurality of independent data units.

### Claim 13 (depends on 8)

The non-transitory computer readable storage medium of claim 8, wherein the aggregation information comprises one of:
a copy of the aggregation indicator; and
a portion of the aggregation indicator.

### Claim 14 (depends on 8)

The non-transitory computer readable storage medium of claim 8, wherein the first computing device is the first DS processing unit.