Relocating data sharing operations for query processing
AssigneeAmazon Technologies, Inc.
InventorsNathan Binkert, Mengchu Cai, Martin Grund, Maor Kleider, Michail Petropoulos, Ippokratis Pandis
Life of this patent
Estimating the term…
Abstract
Relocating data sharing operations for query processing may be implemented when generating plans to perform a query. A query operation that causes a node to share data obtained from a different set of nodes with other nodes in a same set of nodes may be identified. The identified query operation may be replaced with another operation that causes the different set of nodes to share the data directly with the other nodes in the same set of nodes.
Claims20
A system, comprising:
- a memory to store program instructions which, if performed by at least one processor, cause the at least one processor to perform a method to at least:
- receive a query directed to a data set;
- generate a plan to perform processing of the query at a first set of processing nodes and a second set of processing nodes, wherein the plan includes a first operation that instructs one processing node of the first set of processing nodes to send data to another processing node of the first set after the data is received from one or more processing nodes of the second set;
- modify the plan to replace the first operation with a second operation that instructs the one or more processing nodes of the second set to send the data to the other processing node of the first set after the data is obtained from the data set;
- and
- perform the modified plan to process the query.
A method, comprising:
- performing, by one or more computing devices:
- identifying a first query operation that causes a first processing node in a first plurality of processing nodes to receive data from at least one processing node of a second plurality of processing nodes and to cause the data to be shared with at least one other processing node in the first plurality of processing nodes;
- and
- replacing the first query operation with a second query operation, wherein the second query operation causes the data to be sent from the at least one processing node of the second plurality of processing nodes to the first processing node and the at least one other processing node in the first plurality of processing nodes substantially concurrently.
A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:
- identifying a first query operation of a query that causes a first processing node in a first plurality of processing nodes to receive data from at least one processing node of a second plurality of processing nodes and to cause the data to be shared with at least one other processing node in the first plurality of processing nodes as part of a plan to perform the query;
- replacing the first query operation with a second query operation to modify the plan, wherein the second query operation causes the data to be sent from the at least one processing node of the second plurality of processing nodes to the first processing node and the at least one other nodes processing node in the first plurality of processing nodes substantially concurrently;
- and
- causing the first plurality of nodes and the second plurality of nodes to perform the query according to the modified plan.
Dependencies are inferred from the claim text.