Parallel storage system with burst buffer appliance for storage of partitioned key-value store across a plurality of storage tiers

A method comprising:

• communicating, by a burst buffer appliance in a distributed storage system, with a plurality of compute nodes over a network, a first storage tier comprising persistent storage of a first persistent storage type, and a second storage tier comprising persistent storage of a second persistent storage type different than the first persistent storage type, wherein the first storage tier comprises a high-speed memory having a substantially lower access time than the second storage tier;
• storing a plurality of partitions of a partitioned key-value store on one or more of said first storage tier and said second storage tier using a corresponding plurality of partitioned key-value store servers;
• and
• processing, by the burst buffer appliance, one or more tiering commands to store key-value data of the partitioned key-value store on said first and second storage tiers comprising respective disjoint subsets of storage,
• the burst buffer appliance comprising a high-speed memory having a substantially lower access time than the second storage tier and software to access said first storage tier and said second storage tier, wherein the burst buffer appliance is coupled between the plurality of compute nodes and the partitioned key-value store maintained on one or more of said first storage tier and said second storage tier,
• wherein said partitioned key-value store comprises the plurality of partitions stored by the corresponding plurality of partitioned key-value store servers, wherein each partition comprises a distinct range of keys, and wherein a range-knowledgeable client executing in said distributed storage system obtains range information from said corresponding plurality of partitioned key-value store servers in response to a global command, wherein said range information indicates that a given partitioned key-value store server stores a given one of said distinct range of keys, and wherein said range-knowledgeable client sends one or more requests for key-value data within said given distinct range of keys directly to said given partitioned key-value store server.

A partitioned key-value store comprising:

• a first storage tier comprising persistent storage of a first persistent storage type;
• a second storage tier comprising persistent storage of a second persistent storage type different than the first persistent storage type, wherein the first storage tier comprises a high-speed memory having a substantially lower access time than the second storage tier;
• and
• a burst buffer appliance comprising a high-speed memory having a substantially lower access time than the second storage tier and software to access said first storage tier and said second storage tier;
• and
• a plurality of partitioned key-value store servers coupled to the burst buffer appliance via a network, wherein the plurality of partitioned key-value store servers store a corresponding plurality of partitions of the partitioned key-value store on one or more of said first storage tier and said second storage tier;
• wherein the burst buffer appliance communicates with at least one compute node over the network, wherein the burst buffer appliance is coupled between the at least one compute node and the partitioned key-value store maintained on one or more of said first storage tier and said second storage tier,
• the software of the burst buffer appliance processes one or more tiering commands to store key-value data of the partitioned key-value store on said first and second storage tiers comprising respective disjoint subsets of storage,
• wherein said partitioned key-value store comprises the plurality of partitions stored by the corresponding plurality of the partitioned key-value store servers, wherein each partition comprises a distinct range of keys, and wherein a range-knowledgeable client of said partitioned key-value store obtains range information from said corresponding plurality of partitioned key-value store servers in response to a global command, wherein said range information indicates that a given partitioned key-value store server stores a given one of said distinct range of keys, and wherein said range-knowledgeable client sends one or more requests for key-value data within said given distinct range of keys directly to said given partitioned key-value store server.