# US 9,436,723 B2 · Segment deduplication system with compression of segments > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,723 B2 - Document ID: US9436723B2 - Status: Granted - Kind: B2 - Application number: 14/271,246 - Filed: 2014-05-06 - Published: 2014-11-06 - Granted: 2016-09-06 - Earliest claimed: 2009-06-09 - Assignees: EMC Corporation - Applicants: EMC Corporation - Inventors: Christopher R. Lumb - Web page: https://patentgrep.com/patent/US9436723B2 ## CPC - G06F 16/2365: {Ensuring data consistency and integrity} - G06F 3/0608: {Saving storage space on storage systems} - G06F 3/0641: {De-duplication techniques} - G06F 3/0683: {Plurality of storage devices} - G06F 16/10: File systems; File servers - G06F 16/20: of structured data, e.g. relational data - H03M 7/3064: {Segmenting} - G06F 11/1435: {using file system or storage system metadata} - G06F 11/1453: {using de-duplication of the data} - H03M 7/30: Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) ## Abstract A system for storing compressed data comprises a processor and a memory. The processor is configured to receive a compressed segment. The compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments. The processor is further configured to determine whether the compressed segment has been previously stored, and in the event that the compressed segment has not been previously stored, store the compressed segment. The memory is coupled to the processor and configured to provide the processor with instructions. ## Claims ### Claim 1 A system for reading compressed data, comprising: a deduplicated storage device; a processor configured to: receive a request for a data stream, a data block, or a data file to be retrieved; retrieve one or more compressed segments and corresponding metadata information for each compressed segment, wherein the corresponding metadata information comprises a compression type and a compression state from the deduplicated storage device using mapping information, wherein the compression state comprises a state used for compressing the compressed segment, wherein the compression state comprises a 128 bit key, a 256 bit key, or an alphanumeric key, wherein the one or more compressed segments were determined by breaking the data stream, the data block, or the data file into one or more segments and compressing each of the one or more segments, wherein the deduplicated storage device stores compressed segments that have not been previously stored and references to compressed segments that have been determined to be identical to a previously stored compressed segment, wherein the deduplicated storage device stores compressed segments that have not been previously stored with an index entry generated from the compressed segments, wherein the index entry comprises one or more of the following: a digital fingerprint, SHA-1 hash, or Rabin Hash of each unique compressed segment, wherein the deduplicated storage device further retrieves the one or more compressed segments indicated by the mapping information using index entries of compressed segments in the deduplicated storage; decompress the one or more compressed segments using the segment compression information retrieved from the deduplicated storage device, wherein the segment compression information comprises a segment identifier and a compression type; and assemble the one or more compressed segments to reconstruct the data stream, the data block, or the data file using the mapping information, wherein the mapping information comprises a mapping of segment identifiers and the data stream, the data block, or the data file; and a memory coupled to the processor and configured to provide the processor with instructions. ### Claim 2 (depends on 1) A system as in claim 1, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following: a content-based technique or a non-content-based technique. ### Claim 3 (depends on 1) A system as in claim 1, wherein the one or more compressed segments were compressed and decompressed with one of the following: a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system. ### Claim 4 (depends on 1) A system as in claim 1, wherein for a first compressed segment and a second compressed segment that is identical to the first compressed segment, a decompressed first compressed segment is identical to a decompressed second compressed segment. ### Claim 5 A method for reading compressed data, comprising: receiving a request for a data stream, a data block, or a data file to be retrieved; retrieving one or more compressed segments and corresponding metadata information for each compressed segment, wherein the corresponding metadata information comprises a compression type and a compression state from a deduplicated storage device using mapping information, wherein the compression state comprises a state used for compressing the compressed segment, wherein the compression state comprises a 128 bit key, a 256 bit key, or an alphanumeric key, wherein the one or more compressed segments were determined by breaking the data stream, the data block, or the data file into one or more segments and compressing each of the one or more segments, wherein the deduplicated storage device stores compressed segments that have not been previously stored and references to compressed segments that have been determined to be identical to a previously stored compressed segment, wherein the deduplicated storage device stores compressed segments that have not been previously stored with an index entry generated from the compressed segments, wherein the index entry comprises one or more of the following: a digital fingerprint, SHA-1 hash, or Rabin Hash of each unique compressed segment, wherein the deduplicated storage device further retrieves the one or more compressed segments indicated by the mapping information using index entries of compressed segments in the deduplicated storage; decompressing, using a processor, the one or more compressed segments using the segment compression information retrieved from the deduplicated storage device, wherein the segment compression information comprises a segment identifier and a compression type; and assembling the one or more decompressed segments to reconstruct the data stream, the data block, or the data file using the mapping information, wherein the mapping information comprises a mapping of segment identifiers and the data stream, the data block, or the data file. ### Claim 6 (depends on 5) A method as in claim 5, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following: a content-based technique or a non-content-based technique. ### Claim 7 (depends on 5) A method as in claim 5, wherein the one or more compressed segments were compressed and decompressed with one of the following: a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system. ### Claim 8 (depends on 5) A method as in claim 5, wherein for a first compressed segment and a second compressed segment that is identical to the first compressed segment, a decompressed first compressed segment is identical to a decompressed second compressed segment. ### Claim 9 (depends on 5) A system as in claim 5, wherein the deduplicating storage device stores metadata information associated with the segment. ### Claim 10 (depends on 9) A system as in claim 9, wherein the metadata information comprises one or more of the following: the segment identifier, a source identifier, a user identifier, a host identifier, a session identifier, a format type, an associated file, an associated data stream, or an associated data block. ### Claim 11 (depends on 5) A system as in claim 5, wherein the processor is further configured to provide the data stream, the data block, or the data file to a user. ### Claim 12 A computer program product for reading compressed data, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for: receiving a request for a data stream, a data block, or a data file to be retrieved; retrieving one or more compressed segments and corresponding metadata information for each compressed segment, wherein the corresponding metadata information comprises a compression type and a compression state from a deduplicated storage device using mapping information, wherein the compression state comprises a state used for compressing the compressed segment, wherein the compression state comprises a 128 bit key, a 256 bit key, or an alphanumeric key, wherein the one or more compressed segments were determined by breaking the data stream, the data block, or the data file into one or more segments and compressing each of the one or more segments, wherein the deduplicated storage device stores compressed segments that have not been previously stored and references to compressed segments that have been determined to be identical to a previously stored compressed segment, wherein the deduplicated storage device stores compressed segments that have not been previously stored with an index entry generated from the compressed segments, wherein the index entry comprises one or more of the following: a digital fingerprint, SHA-1 hash, or Rabin Hash of each unique compressed segment, wherein the deduplicated storage device further retrieves the one or more compressed segments indicated by the mapping information using index entries of compressed segments in the deduplicated storage; decompressing, using a processor, the one or more compressed segments using the segment compression information retrieved from the deduplicated storage device, wherein the segment compression information comprises a segment identifier and a compression type; and assembling the one or more decompressed segments to reconstruct the data stream, the data block, or the data file using the mapping information, wherein the mapping information comprises a mapping of segment identifiers and the data stream, the data block, or the data file.