Map-reduce job virtualization
AssigneeRed Hat, Inc.
InventorShaun David Appleton
Life of this patent
Estimating the term…
Abstract
A method for map-reduce job virtualization is disclosed. The method includes receiving a map-reduce job written in a first map-reduce language. The map-reduce job is to be performed in parallel on a plurality of nodes of a plurality of clusters. The method also includes selecting one or more clusters to run the map-reduce job. The method further includes identifying a second map-reduce language associated with the selected clusters. The method also includes converting the first map-reduce language of the map-reduce job into the second map-reduce language. The method further causes the map-reduce job in the second map-reduce language to be run on the plurality of nodes of the selected clusters.
Claims20
A method comprising:
- receiving a map-reduce job written in a first map-reduce language, wherein the map-reduce job is to be performed in parallel on a plurality of nodes of a plurality of clusters;
- selecting one or more clusters to run the map-reduce job;
- identifying a second map-reduce language associated with the selected clusters;
- converting the first map-reduce language of the map-reduce job into the second map-reduce language;
- and
- causing the map-reduce job in the second map-reduce language to be run on the plurality of nodes of the selected clusters.
A system comprising:
- a memory to store instructions for a map-reduce management module;
- a processing device, coupled with the memory, to execute the instructions;
- and
- the map-reduce management module, executed by the processing device, to:
- receive a map-reduce job written in a first map-reduce language, wherein the map-reduce job is to be performed in parallel on a plurality of nodes of a plurality of clusters;
- select one or more clusters to run the map-reduce job;
- identify a second map-reduce language associated with the selected clusters;
- convert the first map-reduce language of the map-reduce job into the second map-reduce language;
- and
- cause the map-reduce job in the second map-reduce language to be run on the plurality of nodes of the selected clusters.
A non-transitory computer readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform a method comprising:
- receiving a map-reduce job written in a first map-reduce language, wherein the map-reduce job is to be performed in parallel on a plurality of nodes of a plurality of clusters;
- selecting one or more clusters to run the map-reduce job;
- identifying a second map-reduce language associated with the selected clusters;
- converting the first map-reduce language of the map-reduce job into the second map-reduce language;
- and
- causing the map-reduce job in the second map-reduce language to be run on the plurality of nodes of the selected clusters.
Dependencies are inferred from the claim text.