Method and system for offloading a continuous health-check and reconstruction of data in a data cluster

A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for managing data, the method comprising:

• obtaining, by a storage controller, data from a host;
• applying an erasure coding procedure to the data to obtain a plurality of data chunks and at least one parity chunk;
• deduplicating the plurality of data chunks to obtain a plurality of deduplicated data chunks;
• generating storage metadata associated with the plurality of data chunks and the at least one parity chunk;
• storing, across a plurality of persistent storage devices, the plurality of deduplicated data chunks and the at least one parity chunk, wherein the plurality of persistent storage devices is operatively connected to the storage controller and a second storage controller;
• storing the storage metadata in the storage controller;
• and
• sending a copy of the storage metadata to the second storage controller.

A data cluster, comprising:

• a first storage controller;
• a second storage controller;
• and
• a plurality of persistent storage devices,
• wherein the first storage controller is programmed to:
• obtain data from a host;
• apply an erasure coding procedure to the data to obtain a plurality of data chunks and at least one parity chunk;
• deduplicate the plurality of data chunks to obtain a plurality of deduplicated data chunks;
• generate storage metadata associated with the plurality of data chunks and the at least one parity chunk;
• store, across the plurality of persistent storage devices, the plurality of deduplicated data chunks and the at least one parity chunk;
• store the storage metadata in the first storage controller;
• and
• send a copy of the storage metadata to the second storage controller.