Storage controller, storage device, data processing method, and computer program product

A storage controller comprising:

• a circuitry configured to implement:
• an address generator configured to generate a scan address, which is one of discontinuous addresses indicating storage areas of a first storage that store therein externally written data, according to a particular scan pattern for defining an order of an address of data to be read;
• a reader configured to read data from a storage area of the storage areas of the first storage indicated by a scan address of the scan addresses;
• a duplication detector configured to detect whether the data read by the reader is a duplicate of any one of pieces of data in the first storage by detecting whether a hash value of the data read by the reader is a duplicate of any one of hash values of a past predetermined number of pieces of data, wherein
• the reader reads the data in a unit of a predetermined size from the storage area of the first storage indicated by the scan address;
• the duplication detector includes:
• a hash generator configured to generate the hash value from the data having the predetermined size read by the reader,
• an acquirer configured to acquire location information corresponding to information indicating part of the hash value generated by the hash generator from second information in which information indicating part of the hash value and location information indicating a location in first information including the hash values of the predetermined number of pieces of data are associated with each other, the first information being stored in a second storage, and
• a searcher configured to conduct a linear search from the location in the first information indicated by the location information acquired by the acquirer for a hash value identical to the hash value generated by the hash generator, to detect whether there is a duplicate of the data having the predetermined size read by the reader;
• after the duplication detection by the duplication detector, the address generator updates the scan address according to the particular scan pattern to indicate another storage area of the first storage that is a new target of the duplication detection, the updated scan address being one of the discontinuous addresses,
• the storage controller further comprises a pattern updater configured to perform an update to a different scan pattern from the particular scan pattern after the duplication detector performs duplication detection for data in every storage area of the first storage, the duplication detection been performed using the particular scan pattern, and
• the storage controller further comprises an adder configured to add, when the duplication detector detects that the data read by the reader is not in duplicate, the hash value to the first information.