Back to results
US 8,799,916 B2Granted
Determining an allocation of resources for a job
AssigneeHewlett-Packard Development Company, L. P.
InventorsLudmila Cherkasova, Abhishek Verma
Life of this patent
Estimating the term…
Abstract
A job profile describes characteristics of a job. A performance parameter is calculated based on the job profile, and using a value of the performance parameter, an allocation of resources is determined to assign to the job to meet a performance goal associated with a job.
Claims17
A method comprising:
- receiving, by a system having a processor, a job profile of a job, wherein the job profile describes characteristics of map tasks and reduce tasks, wherein the map tasks produce intermediate results based on segments of input data, and the reduce tasks produce an output based on the intermediate results;
- providing, in the system, a performance model that calculates a performance parameter based on the characteristics of the job profile, a number of the map tasks, a number of the reduce tasks, and an allocation of resources;
- identifying, in the system, plural feasible solutions including corresponding different allocations of resources for which respective values of the performance parameter calculated by the performance model satisfy a performance goal associated with the job, wherein each of the different allocations of resources includes a respective number of map slots in which the map tasks are performed, and a respective number of reduce slots in which the reduce tasks are performed;
- and
- determining, by the system, a particular allocation of resources selected from the plural feasible solutions to assign to the job to meet the performance goal.
2 more elements
An article comprising at least one non-transitory machine-readable storage medium storing instructions that upon execution cause a system having a processor to:
- receive a job profile describing a job to be performed in a distributed computing platform having resources, wherein the job profile includes characteristics of a map stage and a reduce stage of the job, the map stage processing input data to produce an intermediate result, and the reduce stage to process the intermediate result to produce an output;
- use the characteristics of the job profile to calculate corresponding values of a performance parameter for respective different allocations of resources for the job, wherein the performance parameter is computed based on a number of map tasks in the map stage, a number of reduce tasks in the reduce stage, a number of map slots, a number of reduce slots, an average time duration of a map task, an average time duration of a shuffle phase in the reduce stage, an average time duration of a sort phase in the reduce stage, and an average time duration of a reduce phase in the reduce stage;
- and
- determine, based on the values of the performance parameter, a specific allocation of the resources for the job that satisfies a performance goal.
1 more element
A system comprising:
- storage media to store a job profile, wherein the job profile describes a job including a map stage to produce an intermediate result based on input data, and a reduce stage to produce an output based on the intermediate result;
- and
- at least one processor to:
- provide a performance model that calculates a performance parameter based on the job profile, a number of map tasks in the map stage, a number of reduce tasks in the reduce stage, a number of map slots in which the map tasks are performed, a number of reduce slots in which the reduce tasks are performed, an average time duration of a map task, an average time duration of a shuffle phase in the reduce stage, an average time duration of a sort phase in the reduce stage, and an average time duration of a reduce phase in the reduce stage;
- and
- determine, using a value of the performance parameter calculated by the performance model, a particular allocation of resources to assign to the job to meet a performance goal associated with the job.
3 more elements
Dependencies are inferred from the claim text.