# US 2025/0165640 A1 · Method and System for Data Compression and Similarity Detection in a Distributed Storage System > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0165640 A1 - Document ID: US20250165640A1 - Status: Published application - Kind: A1 - Application number: 19/029,011 - Filed: 2025-01-17 - Published: 2025-05-22 - Earliest claimed: 2014-12-31 - Assignees: Pure Storage, Inc. - Applicants: Pure Storage, Inc. - Inventors: Greg R. Dhuse; Andrew D. Baptist; Ilya Volvovski; Ethan S. Wozniak; Jason K. Resch - Web page: https://patentgrep.com/patent/US20250165640A1 ## CPC - G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database} - G06F 3/0604: {Improving or facilitating administration, e.g. storage management} - G06F 3/0622: {in relation to access} - G06F 3/0647: {Migration mechanisms} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems} - G06F 11/1088: {Reconstruction on already foreseen single or plurality of spare disks} - G06F 21/552: {involving long-term monitoring or reporting} - G06F 21/554: {involving event detection and direct action} - G06F 21/64: Protecting data integrity, e.g. using checksums, certificates or signatures - H04L 9/085: {Secret sharing or secret splitting, e.g. threshold schemes} - H04L 9/0891: {Revocation or update of secret information, e.g. encryption key update or rekeying} - H04L 9/0894: {Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage} - H04L 63/10: {for controlling access to devices or network resources} - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04W 12/082: using revocation of authorisation - H04W 12/65: Environment-dependent, e.g. using captured environmental data - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract A method for execution by a storage network begins by receiving a data object for storage, segmenting the data according to a data segmenting protocol to generate a set of data segments and executing a deterministic function on the set of data segments to generate scoring information for respective data segments of the set of data segments. The method continues by comparing the scoring information for a respective data segment to scoring information for previously stored data segments in the storage network and based on the comparison, facilitating storage of a first portion of the set of data segments and not storing a second portion of the set of data segments. ## Claims ### Claim 1 A method for execution by a storage network, the method comprises: receiving a data for storage; segmenting the data according to a data segmenting protocol to generate a set of data segments; executing a deterministic function on the set of data segments to generate scoring information for respective data segments of the set of data segments; comparing the scoring information for the respective data segments to scoring information for one or more previously stored data segments in the storage network; and based on the comparing, facilitating storage of a first portion of the set of data segments and not storing a second portion of the set of data segments. ### Claim 2 (depends on 1) The method of claim 1, wherein facilitating storage of a first portion of the set of data segments includes encoding each data segment of the first portion of the set of data segments based on dispersed encoding parameters to generate a set of encoded data slices. ### Claim 3 (depends on 1) The method of claim 1, wherein the scoring information associated with the respective data segments of the second portion of the set of data segments exceeds a predetermined similarity threshold. ### Claim 4 (depends on 1) The method of claim 1, further comprising: for each data segment of the second portion of data segments, maintaining an index of associated previously stored data segments. ### Claim 5 (depends on 1) The method of claim 1, wherein the deterministic function is a hash function. ### Claim 6 A method for execution by a computing device, the method comprises: changing a decentralized agreement protocol (DAP) of a storage network to a new DAP, wherein storage units of the storage network store encoded data slices; performing, based on the changing, a DAP redistribution operation, wherein the DAP redistribution operation is associated with transfer of affected ones of the encoded data slices from at least one storage unit of the storage units to at least one other storage unit of the storage units; maintaining, based on the changing, at least one source name address map that includes a listing of source names in accordance with storage network addresses for the at least one storage unit, and wherein the source names correspond to the affected ones of the encoded data slices; updating, based on the DAP redistribution operation, the at least one source name address map based on performance of the transfer of the affected ones of the encoded data slices from the at least one storage unit of the storage units to the at least one other storage unit of the storage units; and storing the updated at least one source name address map in a memory of the storage network. ### Claim 7 (depends on 1) The method of claim 1, further comprising, storing the data in a temporary storage before segmenting the data according to a data segmenting protocol. ### Claim 8 (depends on 1) The method of claim 1, wherein the scoring information for a respective data segment includes one or more location identifiers. ### Claim 9 (depends on 1) The method of claim 1, wherein the scoring information for a respective data segment includes one or more location weights. ### Claim 10 (depends on 1) The method of claim 1, wherein the scoring information for a respective data segment is based at least partially on an asset type. ### Claim 11 (depends on 1) The method of claim 1, wherein the scoring information includes any portion of the data segment. ### Claim 12 (depends on 1) The method of claim 1, wherein the scoring information information is sufficient to determine at least one of a data name, a data record identifier, a source name, a slice name, or a plurality of sets of slice names for a respective data segment. ### Claim 13 A computing device of a group of computing devices of a storage network, the computing device comprises: an interface; a local memory; and a processing module operably coupled to the interface and the local memory, wherein the processing module functions to: receive data for storage; segment the data according to a data segmenting protocol to generate a set of data segments; execute a deterministic function on the set of data segments to generate scoring information for respective data segments of the set of data segments; compare the scoring information for the respective data segments to scoring information for one or more previously stored data segments in the storage network; and based on the compare, facilitate storage of a first portion of the set of data segments and not storing a second portion of the set of data segments. ### Claim 14 (depends on 13) The computing device of claim 13, wherein the processing module further functions to: facilitate storage of a first portion of the set of data segments by encoding each data segment of the first portion of the set of data segments based on dispersed encoding parameters to generate a set of encoded data slices. ### Claim 15 (depends on 13) The computing device of claim 13, wherein the scoring information associated with the respective data segments of the second portion of the set of data segments exceeds a predetermined similarity threshold. ### Claim 16 (depends on 13) The computing device of claim 13, wherein the processing module further functions to: for each data segment of the second portion of data segments, maintain an index of associated previously stored data segments. ### Claim 17 (depends on 13) The computing device of claim 13, wherein the deterministic function is a hash function. ### Claim 18 (depends on 13) The computing device of claim 13, wherein the processing module further functions to: store the data in a temporary storage before segmenting the data according to a data segmenting protocol. ### Claim 19 (depends on 13) The computing device of claim 13, wherein the scoring information for a respective data segment includes one or more location identifiers. ### Claim 20 (depends on 13) The computing device of claim 13, wherein the scoring information for a respective data segment is based at least partially on an asset type.