# US 10,372,681 B2 · Tape drive memory deduplication

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,681 B2
- Document ID: US10372681B2
- Status: Granted
- Kind: B2
- Application number: 15/701,994
- Filed: 2017-09-12
- Published: 2019-03-14
- Granted: 2019-08-06
- Earliest claimed: 2017-09-12
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Ole Asmussen; Robert Beiderbeck; Erik Rueger; Markus Schäfer
- Web page: https://patentgrep.com/patent/US10372681B2

## CPC

- G06F 16/1744: {using compression, e.g. sparse files}
- G06F 3/0608: {Saving storage space on storage systems}
- G06F 3/0641: {De-duplication techniques}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 3/0682: {Tape device}
- G06F 11/1453: {using de-duplication of the data}
- G06F 16/1752: {based on file chunks}
- H03M 7/3091: {Data deduplication}
- H03M 7/3095: {using variable length segments}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)

## Abstract

A method and system for improving tape drive memory storage is provided. The method includes receiving, by a storage tape drive, a data stream for storage. The data stream is passed through a non-volatile memory device (NVS2) of the storage tape drive. The data stream is divided into adjacent variable length data chunks and a chunk list file including similarity identifiers for each of the adjacent variable length data chunks is generated and stored within a (non-volatile memory device) NVS1. Duplicate data including duplicated data with respect to a group of data chunks of the adjacent variable length data chunks is identified and deleted from the NVS2 of the storage tape drive such that the group of data chunks remains within NVS2. The group of data chunks is written to a data storage tape cartridge. Pointers identifying each data chunk and an associated storage position are generated and stored.

## Claims

### Claim 1

A tape drive memory storage improvement method comprising:
receiving, by a processor of a storage tape drive hardware device, a data stream, wherein said storage tape drive hardware device internally comprises a deduplication software engine, a first non-volatile memory device (NVS1), a second non-volatile memory device (NVS2), and a first data storage tape cartridge;
passing, by said processor through said NVS2, said data stream;
dividing, by said processor executing said deduplication software engine within said NVS2, said data stream into a plurality of adjacent variable length data chunks;
generating, by said processor, a chunk list file comprising similarity identifiers associated with each of said plurality of adjacent variable length data chunks;
storing, by said processor within said NVS1, said chunk list file;
identifying, by said processor, duplicate data chunks of said plurality of adjacent variable length data chunks, wherein said duplicate data chunks comprise duplicated data with respect to a first group of data chunks of said plurality of adjacent variable length data chunks;
deleting, by said processor from said NVS2, said duplicate data chunks such that said first group of data chunks remain within said NVS2;
writing, by said processor from said NVS2 to said first data storage tape cartridge, said first group of data chunks for storage;
generating, by said processor, pointers identifying each data chunk of said first group of data chunks and an associated storage position, within said first data storage tape cartridge, for each said data chunk of said first group of data chunks;
storing, by said processor, said pointers within said chunk list file located within said NVS1; and
writing, by said processor from said NVS1 to said first data storage tape cartridge, said chunk list file comprising said pointers for storage.

### Claim 2 (depends on 1)

The method of claim 1, wherein said first data storage tape cartridge is removed from said storage tape drive hardware device, and wherein said method further comprises:
deleting, by said processor from said NVS1, said chunk list file comprising said pointers; and
deleting, by said processor from said NVS2, said first group of data chunks.

### Claim 3 (depends on 2)

The method of claim 2, wherein a second data storage tape cartridge is placed within said storage tape drive hardware device, and wherein said method further comprises:
writing, by said processor from said second data storage tape cartridge to said NVS2, a second group of de-duplicated data chunks;
writing, by said processor from said second data storage tape cartridge to said NVS1, an additional chunk list file comprising pointers identifying each data chunk of said second group of de-duplicated data chunks and an associated storage position;
identifying, by said processor based on analysis of said additional chunk list file, necessary duplicate data chunks comprising duplicated data with respect to said second group of de-duplicated data chunks; and
adding, by said processor to said second group of de-duplicated data chunks, said necessary duplicate data chunks such that said second group of de-duplicated data chunks in combination with said necessary duplicate data chunks comprise a complete data file for execution.

### Claim 4 (depends on 3)

The method of claim 3, wherein said second data storage tape cartridge is removed from said storage tape drive hardware device, wherein a third data storage tape cartridge is placed within said storage tape drive hardware device, and wherein said method further comprises:
receiving, by said processor, a second data stream for storage;
executing, by said processor based on said second group of de-duplicated data chunks of said NVS2 and said additional chunk list file of said NVS1, a de-duplication process with respect to said second data stream resulting in a third group of de-duplicated data chunks for storage; and
writing, by said processor to said third data storage tape cartridge, said third group of de-duplicated data chunks for storage.

### Claim 5 (depends on 1)

The method of claim 1, wherein said identifying is executed until a specified data storage size threshold with respect to said data stream has been exceeded.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
partitioning, by said processor said NVS2 such that said data stream is stored within a first partition of said NVS2 and said first group of data chunks is stored within a second partition of said NVS2.

### Claim 7 (depends on 1)

The method of claim 1, wherein said NVS1 comprises a first integrated circuit based memory device, and wherein said NVS2 comprises a second integrated circuit based memory device.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in the storage tape drive hardware device, said code being executed by the processor to implement: said receiving, said storing said data stream, said dividing, said generating said chunk list file, said storing said chunk list file, said identifying, said deleting, said writing said first group of data chunks, said generating said pointers, said storing said pointers, and said writing said chunk list file.

### Claim 9

A computer program product, comprising a computer readable hardware storage device storing a computer readable program code, said computer readable program code comprising an algorithm that when executed by a processor of a storage tape drive hardware device implements a tape drive memory storage improvement method, said method comprising:
receiving, by said processor, a data stream for storage, wherein said storage tape drive hardware device internally comprises a deduplication software engine, a first non-volatile memory device (NVS1), a second non-volatile memory device (NVS2), and a first data storage tape cartridge;
passing, by said processor through said NVS2, said data stream;
dividing, by said processor executing said deduplication software engine within said NVS2, said data stream into a plurality of adjacent variable length data chunks;
generating, by said processor, a chunk list file comprising similarity identifiers associated with each of said plurality of adjacent variable length data chunks;
storing, by said processor within said NVS1, said chunk list file;
identifying, by said processor, duplicate data chunks of said plurality of adjacent variable length data chunks, wherein said duplicate data chunks comprise duplicated data with respect to a first group of data chunks of said plurality of adjacent variable length data chunks;
deleting, by said processor from said NVS2, said duplicate data chunks such that said first group of data chunks remain within said NVS2;
writing, by said processor from said NVS2 to said first data storage tape cartridge, said first group of data chunks for storage;
generating, by said processor, pointers identifying each data chunk of said first group of data chunks and an associated storage position, within said first data storage tape cartridge, for each said data chunk of said first group of data chunks;
storing, by said processor, said pointers within said chunk list file located within said NVS1; and
writing, by said processor from said NVS1 to said first data storage tape cartridge, said chunk list file comprising said pointers for storage.

### Claim 10 (depends on 9)

The computer program product of claim 9, wherein said first data storage tape cartridge is removed from said storage tape drive hardware device, and wherein said method further comprises:
deleting, by said processor from said NVS1, said chunk list file comprising said pointers; and
deleting, by said processor from said NVS2, said first group of data chunks.

### Claim 11 (depends on 10)

The computer program product of claim 10, wherein a second data storage tape cartridge is placed within said storage tape drive hardware device, and wherein said method further comprises:
writing, by said processor from said second data storage tape cartridge to said NVS2, a second group of de-duplicated data chunks;
writing, by said processor from said second data storage tape cartridge to said NVS1, an additional chunk list file comprising pointers identifying each data chunk of said second group of de-duplicated data chunks and an associated storage position; and
identifying, by said processor based on analysis of said additional chunk list file, necessary duplicate data chunks comprising duplicated data with respect to said second group of de-duplicated data chunks; and
adding, by said processor to said second group of de-duplicated data chunks, said necessary duplicate data chunks such that said second group of de-duplicated data chunks in combination with said necessary duplicate data chunks comprise a complete data file for execution.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein said second data storage tape cartridge is removed from said storage tape drive hardware device, wherein a third data storage tape cartridge is placed within said storage tape drive hardware device, and wherein said method further comprises:
receiving, by said processor, a second data stream for storage;
executing, by said processor based on said second group of de-duplicated data chunks of said NVS2 and said additional chunk list file of said NVS1, a de-duplication process with respect to said second data stream resulting in a third group of de-duplicated data chunks for storage; and
writing, by said processor to said third data storage tape cartridge, said third group of de-duplicated data chunks for storage.

### Claim 13 (depends on 9)

The computer program product of claim 9, wherein said identifying is executed until a specified data storage size threshold with respect to said data stream has been exceeded.

### Claim 14 (depends on 9)

The computer program product of claim 9, wherein said method further comprises:
partitioning, by said processor said NVS2 such that said data stream is stored within a first partition of said NVS2 and said first group of data chunks is stored within a second partition of said NVS2.

### Claim 15 (depends on 9)

The computer program product of claim 9, wherein said NVS1 comprises a first integrated circuit based memory device, and wherein said NVS2 comprises a second integrated circuit based memory device.

### Claim 16

A storage tape drive hardware device comprising a processor coupled to a computer-readable memory unit, said memory unit comprising instructions that when executed by the processor implements a tape drive memory storage improvement method comprising:
receiving, by said processor, a data stream for storage, wherein said storage tape drive hardware device internally comprises a deduplication software engine, a first non-volatile memory device (NVS1), a second non-volatile memory device (NVS2), and a first data storage tape cartridge;
passing, by said processor through said NVS2, said data stream;
dividing, by said processor executing said deduplication software engine within said NVS2, said data stream into a plurality of adjacent variable length data chunks;
generating, by said processor, a chunk list file comprising similarity identifiers associated with each of said plurality of adjacent variable length data chunks;
storing, by said processor within said NVS1, said chunk list file;
identifying, by said processor, duplicate data chunks of said plurality of adjacent variable length data chunks, wherein said duplicate data chunks comprise duplicated data with respect to a first group of data chunks of said plurality of adjacent variable length data chunks;
deleting, by said processor from said NVS2, said duplicate data chunks such that said first group of data chunks remain within said NVS2;
writing, by said processor from said NVS2 to said first data storage tape cartridge, said first group of data chunks for storage;
generating, by said processor, pointers identifying each data chunk of said first group of data chunks and an associated storage position, within said first data storage tape cartridge, for each said data chunk of said first group of data chunks;
storing, by said processor, said pointers within said chunk list file located within said NVS1; and
writing, by said processor from said NVS1 to said first data storage tape cartridge, said chunk list file comprising said pointers for storage.

### Claim 17 (depends on 16)

The storage tape drive hardware device of claim 16, wherein said first data storage tape cartridge is removed from said storage tape drive hardware device, and wherein said method further comprises:
deleting, by said processor from said NVS1, said chunk list file comprising said pointers; and
deleting, by said processor from said NVS2, said first group of data chunks.

### Claim 18 (depends on 17)

The storage tape drive hardware device of claim 17, wherein a second data storage tape cartridge is placed within said storage tape drive hardware device, and wherein said method further comprises:
writing, by said processor from said second data storage tape cartridge to said NVS2, a second group of de-duplicated data chunks;
writing, by said processor from said second data storage tape cartridge to said NVS1, an additional chunk list file comprising pointers identifying each data chunk of said second group of de-duplicated data chunks and an associated storage position; and
identifying, by said processor based on analysis of said additional chunk list file, necessary duplicate data chunks comprising duplicated data with respect to said second group of de-duplicated data chunks; and
adding, by said processor to said second group of de-duplicated data chunks, said necessary duplicate data chunks such that said second group of de-duplicated data chunks in combination with said necessary duplicate data chunks comprise a complete data file for execution.

### Claim 19 (depends on 18)

The storage tape drive hardware device of claim 18, wherein said second data storage tape cartridge is removed from said storage tape drive hardware device, wherein a third data storage tape cartridge is placed within said storage tape drive hardware device, and wherein said method further comprises:
receiving, by said processor, a second data stream for storage;
executing, by said processor based on said second group of de-duplicated data chunks of said NVS2 and said additional chunk list file of said NVS1, a de-duplication process with respect to said second data stream resulting in a third group of de-duplicated data chunks for storage; and
writing, by said processor to said third data storage tape cartridge, said third group of de-duplicated data chunks for storage.

### Claim 20 (depends on 16)

The storage tape drive hardware device of claim 16, wherein said identifying is executed until a specified data storage size threshold with respect to said data stream has been exceeded.

### Claim 21

A tape drive memory storage improvement method comprising:
receiving, by a processor of a storage tape drive hardware device, a data file for storage, wherein said storage tape drive hardware device internally comprises a deduplication software engine, a first non-volatile memory device (NVS1), a second non-volatile memory device (NVS2), and a first data storage tape cartridge;
dividing, by said processor executing said deduplication software engine, said data file into a plurality of adjacent variable length data chunks;
identifying, by said processor executing said deduplication software engine, duplicate data chunks of said plurality of adjacent variable length data chunks, wherein said duplicate data chunks comprise duplicated data with respect to a first group of data chunks of said plurality of adjacent variable length data chunks;
storing, by said processor within a first database within said NVS2, said first group of data chunks;
generating, by said processor, pointers identifying each data chunk of said first group of data chunks and an associated storage position, within said first database of said NVS2, for each said data chunk of said first group of data chunks;
storing, by said processor within a second database within said NVS1, said pointers;
first writing, by said processor from said NVS2 to said first data storage tape cartridge, said first group of data chunks for storage; and
second writing, by said processor from said NVS1 to said first data storage tape cartridge, said pointers.

### Claim 22 (depends on 21)

The method of claim 21, further comprising:
before said first writing, encrypting by said processor, said first group of data chunks; and
before said second writing, encrypting by said processor, said pointers.

### Claim 23 (depends on 21)

The method of claim 21, wherein each of said NVS1 and said NVS2 comprises a memory device selected from the group consisting of an integrated circuit based memory device and a removable flash memory device.

### Claim 24

A computer program product, comprising a computer readable hardware storage device storing a computer readable program code, said computer readable program code comprising an algorithm that when executed by a processor of a storage tape drive hardware device implements a tape drive memory storage improvement method, said method comprising:
receiving, by said processor, a data file for storage, wherein said storage tape drive hardware device internally comprises a deduplication software engine, a first non-volatile memory device (NVS1), a second non-volatile memory device (NVS2), and a first data storage tape cartridge;
dividing, by said processor executing said deduplication software engine, said data file into a plurality of adjacent variable length data chunks;
identifying, by said processor executing said deduplication software engine, duplicate data chunks of said plurality of adjacent variable length data chunks, wherein said duplicate data chunks comprise duplicated data with respect to a first group of data chunks of said plurality of adjacent variable length data chunks;
storing, by said processor within a first database within said NVS2, said first group of data chunks;
generating, by said processor, pointers identifying each data chunk of said first group of data chunks and an associated storage position, within said first database of said NVS2, for each said data chunk of said first group of data chunks;
storing, by said processor within a second database within said NVS1, said pointers;
first writing, by said processor from said NVS2 to said first data storage tape cartridge, said first group of data chunks for storage; and
second writing, by said processor from said NVS1 to said first data storage tape cartridge, said pointers.

### Claim 25 (depends on 24)

The computer program product of claim 24, wherein each of said NVS1 and said NVS2 comprises a memory device selected from the group consisting of an integrated circuit based memory device and a removable flash memory device.