Data processing method, device and system of query server

A data processing device, comprising a storage and a processor,

• wherein the processor is electronically coupled to a loading server, the storage comprises instructions that are executable by the processor, and the instructions comprises a first receiving module, an index creation module, and sending module;
• wherein the first receiving module is configured for receiving updated data inserted by the loading server, and the updated data is extracted from at least two table databases stored in a storage server electronically coupled to the loading server,
• wherein the index creation module is configured for creating an inverted index for querying the updated data, and the inverted index is indexed to an index table of the query server;
• wherein the index creation module comprises:
• a workload balancing sub-module, configured for determining one query server to create the inverted index for the updated data according to a workload of each query server in a query server cluster;
• an allocating sub-module, configured for dividing the updated data into a plurality of data partitions and allocating the plurality of data partitions to the one query server;
• and
• a creation sub-module, configured for creating the inverted index for the plurality of data partitions allocated to the query server, and the inverted index is indexed to an index table of the query server;
• wherein the first receiving module is further configured for receiving the plurality of data partitions allocated by the other query servers in the query server cluster, and the plurality of data partitions allocated to the query server is allocated by itself and/or by other query servers in the query server cluster;
• wherein in response to the query server determines that a creation of the inverted index being failed according to the predetermined rule, if the plurality of data partitions corresponding to the failed inverted index is allocated by itself, the sending module is configured for sending a message denoting that the inverted index is failed to create to the loading server which is configured to insert the plurality of data partitions with failed inverted index, and if the plurality of data partitions corresponding to the failed inverted index is allocated by other query server, the sending module is further configured for sending a message denoting that the inverted index is flailed to create to the query server which allocates the plurality of data partitions, and the message comprises an identification of the plurality of data partitions with failed inverted index;
• the first receiving module, is further configured for receiving the plurality of data partitions corresponding to the failed inverted index, and triggering the creation sub-module to create the inverted index for the plurality of data partitions corresponding to the flailed inverted index.

At data processing system, comprising:

• a storage server, a loading server, and a query server cluster, wherein the loading server is electronically coupled to the storage server and the query server cluster,
• wherein the storage server is configured for storing at least two table databases;
• the loading server, is configured for extracting updated data from at least two table databases stored in the storage server, and inserting the updated data into the query server;
• a query server of the query server cluster, is configured for:
• receiving the updated data inserted by the loading server, and determining one query server to create the inverted index for the updated data according to a workload of each query server in the query server cluster;
• dividing the updated data into a plurality of data partitions, and allocating the plurality of data partitions to the one query server;
• in response to the plurality of data partitions being allocated to the query server by itself, creating an inverted index for the plurality of data partitions, wherein the inverted index is indexed to an index table of the query server, and the plurality of data partitions allocated to the query server is allocated by itself and/or by other query servers in the query server;
• determining whether a creation of the inverted index is failed;
• in response to the creation of the inverted index being failed, if the plurality of data partitions corresponding to the failed inverted index is allocated by itself, sending a message denoting that the inverted index is failed to create to the loading server which is configured to insert the plurality of data partitions with failed inverted index, and if the plurality of data partitions corresponding to the flailed inverted index is allocated by other query server a message denoting that the inverted index is failed to create to the query server which allocates the plurality of data partitions, wherein the query server which allocates the plurality of data partitions transfers the message to the loading server, and the message comprises an identification of the plurality of data partitions with failed inverted index;
• and
• receiving the plurality of data partitions corresponding to the filed inverted index, and creating the inverted index for the plurality of data partitions corresponding to the failed inverted index, wherein the inverted index is indexed to the index table.