Scalable partitioning in a multilayered data service framework

A computer system for accessing information about a customer stored in a plurality of customer data stores, wherein each customer data store from the plurality of customer data stores is associated with a distinct customer category, the computer system comprising:

• at least one memory for storing a first customer data store of the plurality of customer data stores,
• wherein each customer data store includes a plurality of partitions,
• wherein each customer data store is associated with one or more customer categories, and
• wherein each customer category from the one or more customer categories is associated with a single customer data store;
• and
• one or more computing devices in communication with the at least one memory, wherein the one or more computing devices are configured to implement a plurality of components, including:
• a customer service component associated with the first customer data store, wherein the customer service component comprises computer-executable code that, when executed by the one or more computing devices, carries out a method comprising:
• receiving a service request to access information about the customer,
• wherein the information about the customer is stored in one of the plurality of partitions of one of the plurality of customer data stores,
• wherein each partition of the plurality of partitions has a plurality of buckets associated therewith, and
• wherein each bucket from the plurality of buckets contains a plurality of partitionable keys related to a subset of customers whose customer information is stored in the partition such that each bucket represents a portion of the customer information stored in the partition;
• and
• a partitioning service component associated with the first customer data store, wherein the partitioning service component comprises computer-executable code that, when executed by the one or more computing devices, carries out a method comprising:
• processing the received service request to identify a partitionable key related to the customer;
• using the identified partitionable key to identify a customer data store of the plurality of customer data stores associated with the customer based at least in part on a customer category from the one or more customer categories associated with the customer;
• and
• if the identified customer data store corresponds to the first data store:
• identifying a bucket of the first customer data store that contains the identified partitionable key;
• performing a bucket mapping using the identified bucket to determine a partition of the first customer data store with which the identified bucket is associated;
• and
• determining whether the portion of the customer information stored in the determined partition represented by the identified bucket is accessible for the received service request;
• if it is determined that the portion of the customer information represented by the identified bucket is accessible for the received service request, sending, by the partitioning service component, to a data partition service configured to manage the determined partition an access request for the information about the customer;
• and
• if the identified customer data store does not correspond to the first customer data store, forwarding the received service request to a partitioning service component associated with the identified customer data store.

A computer-implemented method for accessing information about a customer stored in a plurality of data stores, wherein each data store of the plurality of data stores includes a plurality of partitions, the computer-implemented method comprising:

• under control of instructions that are executed by one or more computing devices:
• receiving, at a first data store of the plurality of data stores, a request to access information about the customer,
• wherein the information about the customer is stored in one of the plurality of partitions of one of the plurality of data stores,
• wherein each partition of the plurality of partitions has at least one set of a plurality of sets of partitionable keys associated therewith,
• wherein each set of partitionable keys from the plurality of sets of partitionable keys includes a plurality of partitionable keys related to a subset of customers whose customer information is stored in a single data store from the plurality of data stores such that a set of partitionable keys represents a portion of the customer information stored in a partition,
• wherein each of the plurality of data stores is associated with one or more customer categories, and
• wherein each customer category from the one or more customer categories is associated with the single data store from the plurality of data stores;
• identifying from the received request a partitionable key from the plurality of partitionable keys related to the subset of customers related to the customer;
• identifying from the identified partitionable key the single data store from the plurality of data stores which contains the identified partitionable key, where the identification is based at least in part on a customer category associated with the customer;
• if the identified single data store corresponds to the first data store:
• using the identified partitionable key to identify the set of partitionable keys within the first data store that includes the identified partitionable key;
• determining, based on the identified set of partitionable keys, the partition within the first data store where the information about the customer is stored;
• determining whether the information about the customer is accessible for the received request based on whether the portion of the customer information that is represented by the identified set of partitionable keys is accessible for the received request;
• and
• if it is determined that the portion of the customer information is accessible for the received request:
• sending to a data partition service that manages the determined partition an access request for the information about the customer;
• and
• obtaining, from the data partition service, a result of the sent access request from the determined partition;
• and
• if the identified single data store does not correspond to the first data store:
• forwarding the received request to access information about the customer to the identified single data store.

A computer system for accessing information about a customer stored in a plurality of customer data stores, wherein each customer data store from the plurality of customer data stores is associated with one or more customer categories and wherein each customer category from the one or more customer categories is associated with a single customer data store, the computer system comprising:

• at least one memory for storing a first customer data store of the plurality of customer data stores,
• wherein each customer data store includes a plurality of partitions, and
• wherein the first customer data store is associated with at least a first customer category from the one or more customer categories;
• and
• one or more computing devices in communication with the at least one memory, wherein the one or more computing devices are configured to implement a plurality of components, including:
• a customer service component associated with the first customer data store, wherein the customer service component comprises computer-executable code that, when executed by the one or more computing devices, carries out a method comprising:
• receiving a service request to access information about the customer,
• wherein the information about the customer is stored in one of one of the plurality of customer data stores,
• wherein each partition of the plurality of partitions has at least one set of a plurality of sets of partitionable keys associated therewith, and
• wherein each set of partitionable keys from the plurality of sets of partitionable key includes a plurality of partitionable keys related to a subset of customers whose customer information is stored in a single customer data store from the plurality of customer data stores such that each set of partitionable keys represents a portion of the customer information stored in the partition;
• and
• a partition partitioning service component associated with the first customer data store, wherein the partitioning service component comprises computer-executable code that, when executed by the one or more computing devices, carries out a method comprising:
• processing the received service request to identify a partitionable key related to the customer;
• using the identified partitionable key to identify a customer data store of the plurality of customer data stores associated with the customer based at least in part on a customer category from the one or more customer categories associated with the customer;
• and
• if the identified customer data store corresponds to the first data store:
• identifying a set of partitionable keys of the first customer data store that includes the identified partitionable key;
• determining, based on the identified set of partitionable keys, a partition of the first customer data store where the information about the customer is stored;
• and
• determining whether the information about the customer is accessible for the received service request based on whether the portion of the customer information that is represented by the identified set of partitionable keys is accessible for the received service request;
• if it is determined that the portion of the customer information is accessible for the received service request, sending, by the partitioning service component, to a data partition service that is configured to manage the determined partition an access request for the information about the customer;
• and
• if the identified customer data store does not correspond to the first customer data store, forwarding the received service request to a partitioning service component associated with the identified customer data store.