Time-based data placement in a distributed storage system

An apparatus for time-based data placement in a distributed storage system, wherein the distributed storage system comprising a plurality of storage nodes for storing user data, and wherein each of at least one user is assigned a storage node sub-set for storing user data of the at least one user, the apparatus comprising:

• a memory, the memory containing computer readable instructions;
• and
• a processor, the processor communicatively coupled to the memory, the processor in response to reading the computer readable instructions configured to:
• monitor and record user accesses on the storage nodes, wherein each storage node includes an identical configuration of storage devices, and
• wherein the monitoring is based on recording information in a three-dimensional time-based array that includes a set of three elements, and wherein each element of the set records a number of user accesses in a corresponding hour;
• calculate a time-based access pattern of a first user according to recorded user accesses of the first user,
• wherein the calculating the time-based access patter of the first user may be performed by a first proxy node separate from the storage nodes that comprise the distributed storage system;
• calculate time-based access patterns of the storage nodes outside the storage node sub-set of the first user in the distributed storage system according to recorded user accesses of the first user,
• wherein the calculating the time-based access patterns of the storage nodes outside the storage node sub-set of the first user may be performed by corresponding modules that are a part of the identical storage nodes that comprise the distributed storage system, and wherein each corresponding module may be configured to form a storage node profile of the corresponding storage node together with the list of all the users having data on the corresponding storage node;
• and
• adjust the storage node sub-set of the first user according to the time-based access pattern of the first user and the time-based access patterns of the storage nodes outside the storage node sub-set of the first user in the distributed storage system, to alleviate I/O hotspots,
• wherein each of the plurality of storage nodes includes hardware having similar input-output performance characteristics to the other storage nodes of the plurality,
• wherein the I/O hotspots are determined based on predetermined I/O upper limits for identifying candidate storage nodes,
• wherein adjusting the storage node sub-set of the first user by the distributed storage system to alleviate I/O hotspots comprises adding, based on the time-based access pattern of the first user and the time-based access patterns of the storage nodes outside the storage node sub-set of the first user, at least one new storage node to the storage node sub-set of the first user, and adding at least one new storage node to the storage node sub-set for each user having data on the storage node to reduce future storage node sub-set adjustments.

A computer program product for time-based data placement in a distributed storage system, wherein the distributed storage system includes a plurality of storage nodes for storing user data, and wherein each of a plurality of users is assigned a storage node sub-set for storing user data, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions configured, when executed by a computer, to cause the computer to perform a method comprising:

• monitoring and recording user accesses on the storage nodes, wherein each of the storage nodes of the distributed storage systems includes hardware having similar input-output performance, wherein each storage node includes an identical configuration of storage devices, and
• wherein the monitoring is based on recording information in a three-dimensional time-based array that includes a set of three elements, and wherein each element of the set records a number of user accesses in a corresponding hour;
• calculating, based on the recorded user accesses of a first user, a time-based access pattern of the first user,
• wherein the calculating the time-based access patter of the first user may be performed by a first proxy node separate from the storage nodes that comprise the distributed storage system;
• calculating, based on the recorded user accesses, time-based access patterns of the storage nodes outside a storage node sub-set of the first user in the distributed storage system,
• wherein the calculating the time-based access patterns of the storage nodes outside the storage node sub-set of the first user may be performed by corresponding modules that are a part of the identical storage nodes that comprise the distributed storage system, and wherein each corresponding module may be configured to form a storage node profile of the corresponding storage node together with the list of all the users having data on the corresponding storage node;
• adding, by the distributed storage system to alleviate I/O hotspots and based on the time-based access pattern of the first user and the time-based access patterns of the storage nodes outside of the storage node sub-set of the first user in the distributed storage system, at least one new storage node to the storage node sub-set of the first user, to alleviate I/O hotspots, wherein the I/O hotspots are determined based on predetermined I/O upper limits for identifying candidate storage nodes;
• and
• adding, by the distributed storage system to alleviate potential I/O hotspots, at least one new storage node to the storage node sub-set for each user other than the first user having data on the storage node.