Assignment of data within file systems
AssigneeInternational Business Machines Corporation
InventorsUttam Jain, Nimrod Megiddo, Umar F. Minhas, Fatma Ozcan, Robbert Van Der Linden
Life of this patent
Estimating the term…
Abstract
The embodiments relate to assigning data to processors of a file system. Metadata associated with respective blocks of data, and an initial batch of the blocks is assigned to nodes of a file system based on the metadata. Unassigned blocks are selectively assigned to one or more of the nodes. The selective assignment includes constructing a linear regression model based on node data, and determining a value for each node based on the linear regression model. Each value is associated with a predicted load corresponding to a new assignment of one or more unassigned blocks.
Claims18
A method comprising:
- assigning an initial batch of blocks of data to one or more nodes of a file system;
- performing a first assignment circuit, including selectively assigning one or more unassigned blocks to the one or more nodes, the selective assignment comprising:
- constructing a linear regression model based on node data;
- and
- for each node, determining a value based on the linear regression model, wherein each value is associated with a predicted load corresponding to the selective assignment of the one or more unassigned blocks;
- and
- in response to determining the predicted load is less than a control factor associated with the selective assignment, assigning the one or more unassigned blocks to the one or more nodes;
- selectively adjusting the control factor, the adjustment in response to determining that the predicted load for the one or more nodes exceed the control factor;
- and
- performing a second assignment circuit, including selectively assigning the one or more unassigned blocks to the one or more nodes exceeding the control factor based on the selectively adjusted control factor.
A computer program product comprising a computer readable hardware storage device having program code embodied therewith, the program code executable by a processing unit to:
- assign an initial batch of blocks of data to one or more nodes of a file system;
- perform a first assignment circuit, including selectively assign one or more unassigned blocks to the one or more nodes, the selective assignment comprising:
- construct a linear regression model based on node data;
- and
- for each node, determine a value based on the linear regression model, wherein each value is associated with a predicted load corresponding to the selective assignment of the one or more unassigned blocks;
- and
- in response to determining the predicted load is less than a control factor associated with the selective assignment, assign the one or more unassigned blocks to the one or more nodes;
- selectively adjust the control factor, the adjustment in response to determining that the predicted load for the one or more nodes exceed the control factor;
- and
- perform a second assignment circuit, including selectively assign the one or more unassigned blocks to the one or more nodes exceeding the control factor based on the selectively adjusted control factor.
A system comprising:
- a processing unit in communication with memory;
- a plurality of nodes operatively coupled to the processing unit, each node having local persistent storage and a local processor;
- and
- a tool in communication with the nodes, the tool configured to:
- assign an initial batch of blocks of data to one or more nodes of a file system;
- perform a first assignment circuit, including selectively assign one or more unassigned blocks to the one or more nodes, the selective assignment comprising the tool to:
- construct a linear regression model based on node data;
- and
- for each node, determine a value based on the linear regression model, wherein each value is associated with a predicted load corresponding to the selective assignment of the one or more unassigned blocks;
- and
- in response to determining the predicted load is less than a control factor associated with the selective assignment, assign the one or more unassigned blocks to the one or more nodes;
- selectively adjust the control factor, the adjustment in response to determining that the predicted load for the one or more nodes exceed the control factor;
- and
- perform a second assignment circuit, including selectively assigning the one or more unassigned blocks to the one or more nodes exceeding the control factor based on the selectively adjusted control factor.
Dependencies are inferred from the claim text.