Performing global computation in distributed database systems

A non-transitory storage device storing instructions that, when executed by one or more computer processors, are operable cause one or more processors to perform operations comprising:

• receiving, by a distributed database system including a cluster of nodes that each correspond to a server, a request to perform an action on the distributed database system, wherein:
• the nodes of the cluster are organized as a plurality of subclusters including a first subcluster and a second subcluster, each subcluster including one or more respective nodes,
• a first calculation master executes on a node in the first subcluster,
• a second calculation master executes on a node in the second subcluster, and
• the action includes a global computation that uses data from each node of the cluster of nodes;
• distributing, by the first calculation master and in response to the request, the global computation into local computations to be performed by respective subclusters;
• generating a first intermediate result comprising a result of first local computation, including performing the first local computation by the first calculation master using first local data from each node in the first subcluster, the first local computation including a portion of the global computation that corresponds to the first subcluster;
• receiving a second intermediate result by the first calculation master from the second calculation master, the second intermediate result comprising a result of second local computation performed by the second calculation master using second local data from each node in the second subcluster, the second local computation including a portion of the global computation that corresponds to the second subcluster;
• performing, by the first computation master, the global computation using the first intermediate result and the second intermediate result;
• and
• performing a database operation that reallocates resources dedicated to processing a query by the nodes of the distributed database system and that is conditioned upon a result of the global computation computed from the intermediate results computed by the nodes of the distributed database system that are processing the query.

A system comprising:

• one or more processors;
• and
• a non-transitory storage device storing instructions that, when executed by one or more computer processors, are operable cause one or more processors to perform operations comprising:
• receiving, by a distributed database system including a cluster of nodes that each correspond to a server, a request to perform an action on the distributed database system, wherein:
• the nodes of the cluster are organized as a plurality of subclusters including a first subcluster and a second subcluster, each subcluster including one or more respective nodes,
• a first calculation master executes on a node in the first subcluster,
• a second calculation master executes on a node in the second subcluster, and
• the action includes a global computation that uses data from each node of the cluster of nodes;
• distributing, by the first calculation master and in response to the request, the global computation into local computations to be performed by respective subclusters;
• generating a first intermediate result comprising a result of first local computation, including performing the first local computation by the first calculation master using first local data from each node in the first subcluster, the first local computation including a portion of the global computation that corresponds to the first subcluster;
• receiving a second intermediate result by the first calculation master from the second calculation master, the second intermediate result comprising a result of second local computation performed by the second calculation master using second local data from each node in the second subcluster, the second local computation including a portion of the global computation that corresponds to the second subcluster;
• performing, by the first computation master, the global computation using the first intermediate result and the second intermediate result;
• and
• performing a database operation that reallocates resources dedicated to processing a query by the nodes of the distributed database system and that is conditioned upon a result of the global computation computed by the nodes of the distributed database system that are processing the query.