Method and apparatus for data partitioning and replication in a searchable data service

A method, comprising:

• processing query requests received from client applications by a searchable data service implemented on a plurality of computer devices, wherein the searchable data service is configured to locate searchable data service objects in respective searchable indexes that satisfy the query requests, wherein each searchable data service object specifies two or more attributes of a particular entity in a particular one of a plurality of independent data stores on one or more storage devices on a network and separate from the plurality of computer devices that implement the searchable data service, wherein each searchable index stores searchable data service objects for a particular one of the plurality of independent data stores, and wherein the attributes include a unique entity identifier for locating the particular entity in the particular data store;
• wherein the searchable data service comprises a plurality of storage nodes each configured to locally store one or more partitions of one of the plurality of searchable indexes, wherein each partition is configured to store a set of the searchable data service objects stored in the respective searchable index;
• storing on one of the plurality of storage nodes a particular partition of the searchable index, wherein the particular partition is configured to persistently store a particular set of the searchable data service objects stored in the respective searchable index;
• and
• repartitioning the particular partition on the storage node to generate two or more new partitions on the storage node, wherein each of the two or more new partitions includes a different subset of the set of searchable data service objects stored in the particular partition;
• wherein each partition of the searchable index is movable to a different storage node;
• and
• wherein each partition of the searchable index is replicable one or more others of the storage nodes.

A computer-accessible memory medium storing program instructions, wherein the program instructions are computer-executable to implement a searchable data service on a plurality of nodes, wherein the searchable data service is configured to:

• process query requests received from client applications on the plurality of nodes to locate searchable data service objects in respective searchable indexes that satisfy the query requests, wherein each searchable data service object specifies two or more attributes of a particular entity in a particular one of a plurality of independent data stores on one or more storage devices on a network and separate from computer devices that implement the searchable data service, wherein each searchable index stores searchable data service objects for a particular one of the plurality of independent data stores, and wherein the attributes include a unique entity identifier for locating the particular entity in the particular data store;
• wherein the searchable data service comprises a plurality of storage nodes, wherein each storage node is configured to:
• persistently store on the storage node a partition of one of the plurality of searchable indexes, wherein the partition is configured to store a set of the searchable data service objects stored in the respective searchable index;
• and
• repartition the partition to generate two or more new partitions on the storage node, wherein each of the two or more new partitions includes a different subset of the set of searchable data service objects stored in the original partition;
• wherein each partition of the searchable index is movable to a different storage node;
• and
• wherein each partition of the searchable index is replicable to one or more others of the storage nodes.

A searchable data service storage node, comprising:

• one or more processors;
• a network interface configured to communicatively couple to a network;
• and
• a memory comprising program instructions, wherein the program instructions are executable by the one or more processors to:
• persistently store a partition of a particular one of a plurality of searchable indexes maintained by a searchable data service implemented on a plurality of computer devices, wherein the searchable data service storage node is implemented on one of the computer devices, wherein the particular searchable index corresponds to a particular one of a plurality of independent data stores on one or more storage devices on the network and separate from the plurality of computer devices that implement the searchable data service, and wherein the partition is configured to store a set of searchable data service objects stored in the particular searchable index;
• process query requests received from client applications via the network to locate searchable data service objects that satisfy the query requests in the particular searchable index, wherein each searchable data service object specifies two or more attributes of a particular entity in the data store corresponding to the particular searchable index, and wherein the attributes include a unique entity identifier for locating the particular entity in the corresponding data store;
• and
• repartition the partition to generate two or more new partitions on the searchable data service storage node, wherein each of the two or more new partitions includes a different subset of the set of searchable data service objects stored in the original partition;
• wherein each partition of the searchable index is movable to different searchable data service storage nodes of the searchable data service;
• and
• wherein each partition of the searchable index is replicable to one or more other searchable data service storage nodes of the searchable data service.