Method and System for Data Compression and Similarity Detection in a Distributed Storage System

A method for execution by a computing device, the method comprises:

• changing a decentralized agreement protocol (DAP) of a storage network to a new DAP, wherein storage units of the storage network store encoded data slices;
• performing, based on the changing, a DAP redistribution operation, wherein the DAP redistribution operation is associated with transfer of affected ones of the encoded data slices from at least one storage unit of the storage units to at least one other storage unit of the storage units;
• maintaining, based on the changing, at least one source name address map that includes a listing of source names in accordance with storage network addresses for the at least one storage unit, and wherein the source names correspond to the affected ones of the encoded data slices;
• updating, based on the DAP redistribution operation, the at least one source name address map based on performance of the transfer of the affected ones of the encoded data slices from the at least one storage unit of the storage units to the at least one other storage unit of the storage units;
• and
• storing the updated at least one source name address map in a memory of the storage network.

A computing device of a group of computing devices of a storage network, the computing device comprises:

• an interface;
• a local memory;
• and
• a processing module operably coupled to the interface and the local memory, wherein the processing module functions to:
• receive data for storage;
• segment the data according to a data segmenting protocol to generate a set of data segments;
• execute a deterministic function on the set of data segments to generate scoring information for respective data segments of the set of data segments;
• compare the scoring information for the respective data segments to scoring information for one or more previously stored data segments in the storage network;
• and
• based on the compare, facilitate storage of a first portion of the set of data segments and not storing a second portion of the set of data segments.