Storage appliance and method of segment deduplication

A storage appliance comprising:

• an interface that provides access to at least one virtual tape drive for at least one host computer;
• a first database that stores metadata about virtual tape volumes received via the interface,
• at least one volatile memory;
• and
• a deduplication engine that deduplicates storage objects;
• wherein each virtual tape volume is represented within the storage appliance as an ordered set of segments of data and the storage appliance is configured to perform the following steps:
• providing, from a volatile memory, at least one buffer memory to store a segment of data;
• receiving a data stream representing a virtual tape volume of the at least one virtual tape drive at the interface;
• filling the at least one buffer memory with data from the received data stream until the received data stream is closed, and a synchronization point of the virtual tape volume is identified in the received data stream or a predefined amount of data has been stored in the buffer memory;
• deduplicating with the deduplication engine the segment of data stored in the at least one buffer memory;
• and
• storing the deduplicated segment of data in a non-volatile storage device;
• wherein data of an existing virtual tape volume is modified by:
• identifying an index of a first segment of data to be modified;
• reading and re-duplicating the deduplicated first segment from the deduplication engine into a memory buffer;
• invalidating all segments of data having an index equal to or greater than the identified index and deleting the corresponding indices from the ordered set;
• creating a new segment of data based on the buffered first segment and a modification request received from the interface;
• deduplicating and storing the new segment;
• and
• adding the index of the new segment to the end of the ordered set.

A method of segmenting a virtual tape volume provided by an application via an interface to a storage virtualization appliance, the storage virtualization appliance comprising a deduplication engine and a volatile memory, wherein each virtual tape volume is represented within the storage virtualization appliance as an ordered set of segments of data, the segmentation of the virtual tape volume is based on a tape semantic of the interface of the storage virtualization appliance and is performed by repeating the following steps:

• defining a maximum segment size based on an amount of main memory available for a computer system performing the segmentation;
• providing, from the volatile memory, at least one buffer memory of the defined maximum segment size to store a segment of data of the virtual tape volume;
• filling the at least one memory buffer with data of the virtual tape volume received from the application until a segment of the virtual tape volume is considered complete, wherein the segment is considered complete when the at least one memory buffer has been filled to the defined maximum segment size such that the at least one memory buffer is completely filled, when the application writes a file mark serving as an synchronization point or when the virtual tape volume is closed by the application;
• passing a segment of the virtual tape volume for storage to the deduplication engine when the segment is considered complete, and
• processing, by the deduplication engine, the passed segment of the virtual tape volume directly from the at least one memory buffer for deduplicated storage.