Joining data across a parallel database and a distributed processing system

A computer program product for joining data across a parallel database and a distributed processing system, the computer program product comprising:

• a tangible storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising:
• receiving a query on data stored in the parallel database T and data stored in the distributed processing system L;
• applying local projection and one or more local predicates σT based on the query to data stored on the parallel database T to create a filtered data T′;
• applying local projection and one or more local predicates σL based on the query to data stored on the distributed processing system L to create a filtered data L′;
• based on determining that a size of L′ is less than a size of T′ and that the size of L′ is less than a parallel database broadcast threshold, one or more of the nodes distributed processing system transmitting L′ to the parallel database, and in response to receiving L′, one or more nodes of the parallel database executing a join between T′ and L′;
• and
• based on determining that a number of the nodes of distributed processing system n multiplied by the size of T′ is less than the size of L′ and that the size of T′ is less than a distributed files system transfer threshold;
• one or more of parallel database nodes transmitting T′ to the distributed processing system and in response to receiving T′, one or more nodes of the distributed processing system executing a join between T′ and L′.

A computer system for joining data across a parallel database and a distributed processing system, the computer program product comprising:

• a processor configured for executing instructions to perform a method comprising:
• receiving a query on data stored in the parallel database T and the data stored in distributed processing system L;
• applying one or more local predicates σT based on the query to data stored on the parallel database T to create a filtered data T′;
• applying one or more local predicates σL based on the query to data stored on the distributed processing system L to create a filtered data L′;
• based on determining that a size of L′ is less than a size of T′ and that the size of L′ is less than a parallel database broadcast threshold, one or more of the nodes of distributed processing system transmitting L′ to the parallel database, and in response to receiving L′, one or more nodes of the parallel database executing a join between T′ and L′;
• and
• based on determining that a number of the nodes distributed processing system n multiplied by the size of T′ is less than the size of L′ and that the size of T′ is less than a distributed files system transfer threshold;
• one or more of parallel database nodes transmitting T′ to the distributed processing system and in response to receiving T′, one or more nodes of the distributed processing system executing a join between T′ and L′.