System and method for supporting persistence partition discovery in a distributed data grid

A system for recovering persisted backups of in-memory data partitions, the system comprising:

• a plurality of respective members of an in-memory data grid that each execute on a physical computer;
• for each respective member of the in-memory data grid, a respective in-memory data partition in a volatile memory of the physical computer on which the respective member executes, wherein each respective member of the in-memory data grid, executes a cache service utilizing the respective in-memory data partition in the volatile memory of the physical computer on which the respective member executes;
• wherein each respective in-memory data partition is persisted, as a plurality of persisted partitions, to non-volatile storage that is accessible to the respective member of each respective partition;
• wherein subsequent to an event where a plurality of the respective in-memory data partitions are lost from the volatile memory of the physical computer on which the respective members of the respective in-memory data partitions which are lost execute, and wherein, upon a recovery effort by a coordinating member of the distributed in-memory data grid, only a subset of the plurality of the respective members of the in-memory data grid, which subset is an active member plurality, are available, the system operates to:
• determine, by the coordinating member of the distributed in-memory data grid, a list of each persisted partition in the plurality of persisted partitions;
• determine, by the coordinating member from the list of each persisted partition in the plurality of persisted partitions, which members of the active member plurality has access to each persisted partition in the plurality of persisted partitions via access to the non-volatile storage to which each of the plurality of persisted partitions is persisted;
• determine, by the coordinating member, a prospective view of ownership of each persisted partition in the plurality of persisted partitions based on the determined access of members of the active member plurality to each persisted partition in the plurality of persisted partitions, wherein the prospective view of ownership assigns ownership of each of the persisted partitions in the plurality of persisted partitions to one of the members of the active member plurality;
• provide, by the coordinating member, each member of the active member plurality with a recovery assignment based on the prospective view of ownership of each respective in-memory data partition;
• and
• wherein each member of the active member plurality, carries out the recovery assignment, wherein carrying out the recovery assignment by each member of the active member plurality includes recovering, from the non-volatile storage and to the volatile memory of the physical computer on which the member of the active member plurality executes, the persisted partition to which the member was determined to have access and to which the prospective view of ownership assigned ownership, and promoting the recovered persisted partition to a primary in-memory data partition of the member, wherein each respective member of the active member plurality executes the cache service utilizing the respective in-memory data partition which was recovered from the non-volatile storage to the volatile memory of the physical computer on which the respective member of the active member plurality executes.

A non-transitory machine readable storage medium having instructions stored thereon for recovering persisted backups of in-memory data partitions, which instructions, when executed, configure one or more computers to perform steps comprising:

• providing a plurality of respective members of an in-memory data grid that each execute on a physical computer;
• for each respective member of the in-memory data grid, providing a respective in-memory data partition in a volatile memory of the physical computer on which the respective member executes, wherein each respective member of the in-memory data grid, executes a cache service utilizing the respective in-memory data partition in the volatile memory of the physical computer on which the respective member executes;
• persisting, as a plurality of persisted partitions, each respective in-memory data partition to non-volatile storage that is accessible to the respective member of each respective partition;
• subsequent to an event where a plurality of the respective in-memory data partitions are lost from the volatile memory of the physical computer on which the respective members of the respective in-memory data partitions which are lost execute, and wherein, upon a recovery effort by a coordinating member of the distributed in-memory data grid, only a subset of the plurality of the respective members of the in-memory data grid, which subset is an active member plurality, are available:
• determining, by the coordinating member of the distributed in-memory data grid, a list of each persisted partition in the plurality of persisted partitions;
• determining, by the coordinating member from the list of each persisted partition in the plurality of persisted partitions, which members of the active member plurality has access to each persisted partition in the plurality of persisted partitions via access to the non-volatile storage to which each of the plurality of persisted partitions is persisted;
• determining, by the coordinating member, a prospective view of ownership of each persisted partition in the plurality of persisted partitions based on the determined access of members of the active member plurality to each persisted partition in the plurality of persisted partitions, wherein the prospective view of ownership assigns ownership of each of the persisted partitions in the plurality of persisted partitions to one of the members of the active member plurality;
• providing, by the coordinating member, each member of the active member plurality with a recovery assignment based on the prospective view of ownership of each respective in-memory data partition;
• and
• carrying out, by each member of the active member plurality, the recovery assignment, wherein carrying out the recovery assignment by each member of the active member plurality includes recovering, from the non-volatile storage and to the volatile memory of the physical computer on which the member of the active member plurality executes, the persisted partition to which the member was determined to have access and to which the prospective view of ownership assigned ownership, and promoting the recovered persisted partition to a primary in-memory data partition of the member, wherein each respective member of the active member plurality executes the cache service utilizing the respective in-memory data partition which was recovered from the non-volatile storage to the volatile memory of the physical computer on which the respective member of the active member plurality executes.