System and method for consistency checks in cloud object stores using microservices

A system for detecting an inconsistency between data stored in cloud storage and used in a deduplication backup platform having a backup server, comprising:

• a first processing component locating metadata objects and data objects in the cloud through a cloud-based consistency check process, wherein metadata objects reference data objects by ID and data objects contain actual data;
• local storage of the backup server storing the metadata objects;
• cloud storage storing mirrored versions of the metadata objects and the data objects;
• a consistency check processing component determining a current prefix, wherein a prefix comprises a first number of characters of an ID and defines a range to process the metadata objects, listing all data objects that have the same prefix as the current prefix, in a list, determining, for all metadata objects, whether corresponding referred data blocks are available in the list, and marking unavailable referred data blocks as inconsistent data;
• and
• in interface transmitting the marked inconsistent data to an analytics process of the system for further analysis of cause and solution of the detected inconsistency.

A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to perform a method of detecting an inconsistency between data stored in cloud storage and used in a deduplication backup system having a backup server, comprising:

• locating metadata objects and data objects in the cloud through a cloud-based consistency check process, wherein metadata objects reference data objects by ID and data objects contain actual data;
• storing the metadata objects in both storage local to the backup server and the cloud storage, and storing the data objects only in the cloud storage;
• determining a current prefix, wherein a prefix comprises a first number of characters of an ID and defines a range to process the metadata objects;
• listing, in a list, all data objects that have the same prefix as the current prefix;
• determining, for all metadata objects, whether corresponding referred data blocks are available in the list;
• and
• marking unavailable referred data blocks as inconsistent data.