Computing cross products using map reduce
AssigneeRoblox Corporation
InventorsAswath Manoharan, Nikolaus Sonntag
Life of this patent
Estimating the term…
Abstract
A request to perform a set of map reduce jobs that generate a plurality of cross products from a dataset based on a join field is received. The join field indicates that each of the plurality of cross products is to be generated from a corresponding subset of the dataset. Responsive to receiving the request to perform the set of map reduce jobs, the set of map reduce jobs on the dataset to generate the plurality of cross products is performed. The final output data of the set of map reduce jobs is stored. The final output data comprises the plurality of cross products.
Claims20
A method, comprising:
- receiving, by a processing device, a request to perform a set of map reduce jobs that generate a plurality of cross products from a dataset based on a join field, wherein the join field indicates that each of the plurality of cross products is to be generated from data of a single subset of a plurality of subsets of the dataset;
- responsive to receiving the request to perform the set of map reduce jobs, performing the set of map reduce jobs on the dataset to generate final output data comprising the plurality of cross products, wherein performing the set of map reduce jobs comprises distributing map reduce operations of the set of map reduce jobs to respective nodes for processing in parallel;
- and
- storing the final output data of the set of map reduce jobs.
A system comprising:
- a memory;
- and
- a processing device, coupled to the memory, to:
- receive a request to perform a set of map reduce jobs that generate a plurality of cross products from a dataset based on a join field, wherein the join field indicates that each of the plurality of cross products is to be generated from data of a single subset of a plurality of subsets of the dataset;
- responsive to receiving the request to perform the set of map reduce jobs, perform the set of map reduce jobs on the dataset to generate final output data comprising the plurality of cross products, wherein to perform the set of map reduce jobs comprises map reduce operations of the set of map reduce jobs to respective nodes for processing in parallel;
- and
- store the final output data of the set of map reduce jobs.
A non-transitory computer-readable medium comprising instructions that, responsive to execution by a processing device, cause the processing device to perform operations comprising:
- receiving, by the processing device, a request to perform a set of map reduce jobs that generate a plurality of cross products from a dataset based on a join field, wherein the join field indicates that each of the plurality of cross products is to be generated from data of a single subset of a plurality of subsets of the dataset;
- responsive to receiving the request to perform the set of map reduce jobs, performing the set of map reduce jobs on the dataset to generate final output data comprising the plurality of cross products, wherein performing the set of map reduce jobs comprises distributing map reduce operations of the set of map reduce jobs to respective nodes for processing in parallel;
- and
- storing the final output data of the set of map reduce jobs.
Dependencies are inferred from the claim text.