# US 10,769,111 B2 · Data transformation of Cassandra files for improved deduplication during backup > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,111 B2 - Document ID: US10769111B2 - Status: Granted - Kind: B2 - Application number: 15/961,794 - Filed: 2018-04-24 - Published: 2019-10-24 - Granted: 2020-09-08 - Earliest claimed: 2018-04-24 - Assignees: EMC Holding Company, LLC - Applicants: EMC IP Holding Company LLC - Inventors: Charles Christopher Bailey; Donna Barry Lewis; Jeffrey Ford; Frederick Douglis - Web page: https://patentgrep.com/patent/US10769111B2 ## CPC - G06F 16/1748: {De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641)} - G06F 3/0608: {Saving storage space on storage systems} - G06F 11/1453: {using de-duplication of the data} - G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors - G06F 16/2282: {Tablespace storage structures; Management thereof} - G06F 16/258: {Data format conversion from or to a database} - G06F 40/205: Parsing - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Cassandra SSTable data is transformed to provide data rows that are a consistent size such that data in each row has a length that is contained within a selected fixed sized kilobyte segment for deduplication. Tables of a Cassandra cluster node are translated in parallel to JSON format using Cassandra SSTableDump and the table rows are parsed to provide data rows corresponding to the data in each table row. Each row of data is padded with a predictable pattern of bits such that the data row has a length corresponding to the selected fixed segment size and has boundary locations that correspond to multiple of the selected segment size. Since each row of data starts on a segment boundary, duplicate rows of data will be identified wherever they move within a table. ## Claims ### Claim 1 A method of improving deduplication and backup of data files that store changing data in tables as a plurality of rows of variable length data that are reordered and merged with rows as data changes, comprising: selecting a fixed segment size of data as a data unit for deduplication; translating each row of data in a selected table into a format that enables access to the data in each said row in said selected table; parsing the data in said translated rows of said selected table to provide row lengths that are sized to fit within multiples of said fixed segment size; adjusting the row length of each parsed row in said selected table to have a length that corresponds to said fixed segment size by padding each parsed row with a predetermined predictable pattern of bytes to produce a transformed table having rows of the same size; and deduplicating individual rows of data in said transformed table to identify data that is a duplicate of previously stored data. ### Claim 2 (depends on 1) The method of claim 1, wherein each row of data has a unique key value, and data rows are ordered within said selected table by said key values. ### Claim 3 (depends on 1) The method of claim 1, wherein said translating comprises translating the data in the selected table into a format such that the data in each row is contained within a length corresponding to said fixed segment size. ### Claim 4 (depends on 1) The method of claim 1, wherein said adjusting comprises padding the length of each row of said selected table such that the boundaries of each such row are on locations in said selected table corresponding to multiples of said selected segment size. ### Claim 5 (depends on 1) The method of claim 1, wherein said tables are Cassandra tables and wherein said translating, said parsing and said adjusting are performed in parallel on a plurality of said Cassandra tables. ### Claim 6 (depends on 5) The method of claim 5, wherein the Cassandra tables are located on one or more nodes of a Cassandra cluster of nodes, and said method comprises transferring said transformed table over a network to another cluster for said deduplicating. ### Claim 7 (depends on 6) The method of claim 6 further comprising generating a fingerprint for each row of data, and transferring said fingerprints over said network for comparison with stored fingerprints of previous data to identify a row of duplicate data before transferring said row of duplicate data over said network. ### Claim 8 Non-transitory computer readable storage medium storing executable instructions for controlling a computer to perform a method of deduplication and backup of data files that store changing data in tables as a plurality of rows of variable length data that are reordered and merged with rows as data changes, comprising: selecting a fixed segment size of data as a data unit for deduplication; translating of each row of data in a selected table into a format that enables access to the data in each row in said selected table; parsing the data in said translated rows of said selected table to provide row lengths that are sized to fit within multiples of said fixed segment size; adjusting the row length of each parsed row in said selected table to have a length that corresponds to said fixed segment size by padding each parsed row with a predetermined predictable pattern of bytes to produce a transformed table having rows of data of the same size; and deduplicating individual rows of data in said transformed table to identify data that is a duplicate of previously stored data. ### Claim 9 (depends on 8) The non-transitory computer readable storage medium of claim 8, wherein each row of data has a unique key value, and data rows are ordered within said selected table by said key values. ### Claim 10 (depends on 8) The non-transitory computer readable storage medium of claim 8, wherein said translating comprises translating the data in the selected table into a format such that the data in each row is contained within a length corresponding to said fixed segment size. ### Claim 11 (depends on 8) The non-transitory computer readable storage medium of claim 8, wherein said adjusting comprises padding the length of each row of said selected table such that the boundaries of each such row are on locations in said selected table corresponding to multiples of said selected segment size. ### Claim 12 (depends on 8) The non-transitory computer readable storage medium of claim 8, wherein said tables are Cassandra tables and wherein said translating, said parsing and said adjusting are performed in parallel on a plurality of said Cassandra tables. ### Claim 13 (depends on 12) The non-transitory computer readable storage medium of claim 12, wherein the Cassandra tables are located on one or more nodes of a Cassandra cluster of nodes, and said method comprises transferring said transformed table over a network to another cluster for said deduplicating. ### Claim 14 (depends on 13) The non-transitory computer readable storage medium of claim 13, further comprising generating a fingerprint for each row of data, and transferring said fingerprints over said network for comparison with stored fingerprints of previous data to identify a row of duplicate data before transferring said row of duplicate data over said network.