Customer-related partitioning of journal-based storage systems

A method, comprising:

• performing, by a control plane component implemented at one or more computing devices of a distributed storage system:
• receiving, from a client via a programmatic interface provided by the control plane component, a request indicating partitioning instructions for materialization of writes indicated in committed transaction entries of a journal and directed to a first data object of the distributed storage system;
• configuring, responsive to the request indicating the partitioning instructions for the materialization of the writes indicated in the committed transaction entries of the journal and directed to the first data object, a plurality of materialization nodes and write appliers for respective partitions of the first data object in accordance with the partitioning instructions, including a first materialization node and a first write applier configured by the control plane to apply writes from the journal to a first partition of the first data object at the first materialization node and a second materialization node and a second write applier configured by the control plane to apply writes from the journal to a second partition of the first data object at the second materialization node,
• wherein the first write applier for the first partition of the first data object is configured by the control plane to apply writes from the journal at the first materialization node and is configured to identify and select, according to the partitioning instructions received from the client, a first set of writes corresponding to the first partition of the first data object from among a plurality of sets of writes of the committed transaction entries of the journal that correspond to different partitions of the first data object, and apply, by the first write applier, the first set of writes that were identified and selected from the journal by the first write applier to the first partition of the first data object at the first materialization node, and
• wherein the second write applier for the second partition of the first data object is configured by the control plane to apply writes from the journal at the second materialization node and is configured to identify and select, according to the partitioning instructions received from the client, a second set of writes corresponding to the second partition of the first data object from among the plurality of sets of writes of the committed transaction entries of the journal that correspond to the different partitions of the first data object, and apply, by the second write applier, the second set of writes that were identified and selected from the journal by the second write applier to the second partition of the first data object at the second materialization node,
• wherein the first set of writes and the second set of writes indicated in the committed transaction entries of the journal are to be respectively applied by different write appliers to different partitions of the first data object at different materialization nodes based on the request received from the client;
• and
• sending an acknowledgement of the request indicating the partitioning instructions for the materialization of the writes indicated in the committed transaction entries of the journal and directed to the first data object to the client.

One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors implement a control plane component of a distributed storage system, the control plane component to:

• receive, from a client via a programmatic interface provided by the control plane component, a request indicating partitioning instructions for materialization of writes indicated in committed transaction entries of a journal and directed to a first data object of the distributed storage system;
• configure, responsive to the request indicating the partitioning instructions for the materialization of the writes indicated in the committed transaction entries of the journal and directed to the first data object, a plurality of materialization nodes and write appliers for respective partitions of the first data object in accordance with the partitioning instructions, including a first materialization node and a first write applier configured by the control plane to apply writes from the journal to a first partition of the first data object at the first materialization node and a second materialization node and a second write applier configured by the control plane to apply writes from the journal to a second partition of the first data object at the second materialization node,
• wherein the first write applier for the first partition of the first data object is configured by the control plane to apply writes from the journal at the first materialization node and is configured to identify and select, according to the partitioning instructions received from the client, a first set of writes corresponding to the first partition of the first data object from among a plurality of sets of writes of the committed transaction entries of the journal that correspond to different partitions of the first data object, and apply, by the first write applier, the first set of writes that were identified and selected from the journal by the first write applier to the first partition of the first data object at the first materialization node, and
• wherein the second write applier for the second partition of the first data object is configured by the control plane to apply writes from the journal at the second materialization node and is configured to identify and select, according to the partitioning instructions received from the client, a second set of writes corresponding to the second partition of the first data object from among the plurality of sets of writes of the committed transaction entries of the journal that correspond to the different partitions of the first data object, and apply, by the second write applier, the second set of writes that were identified and selected from the journal by the second write applier to the second partition of the first data object at the second materialization node,
• wherein the first set of writes and the second set of writes indicated in the committed transaction entries of the journal are to be respectively applied by different write appliers to different partitions of the first data object at different materialization nodes based on the request received from the client;
• and
• send an acknowledgement of the request indicating the partitioning instructions for the materialization of the writes indicated in the committed transaction entries of the journal and directed to the first data object to the client.