Temporal difference learning, reinforcement learning approach to determine optimal number of threads to use for file copying
AssigneeMicro Focus Software Inc.
InventorArshad Javeed
Life of this patent
Estimating the term…
Abstract
For a given file type, an optimal number of threads to use to copy files of each of a number of different discrete file sizes is determined, using a temporal difference learning, reinforcement learning approach in which file copy time is used as feedback reward reinforcement. A continuous function corresponding to the given file type and outputting the number of threads to use to copy files having this given file type and that are of any input file size is fitted onto the optimal numbers of threads determined for the discrete file sizes.
Claims20
A method comprising:
- determining a continuous function specific to a given file type and that outputs a number of threads to use to copy an input file of any input file size that has the given file type, the continuous function calculating the number of threads to use to copy the input file based on a file size of the input file, wherein determining the continuous function comprising:
- for each respective discrete file size of a plurality of discrete file sizes that are each a specific different file size, determining, without using the continuous function, an optimal number of threads to use to copy files of the respective discrete file size and having the given file type, using a temporal difference learning, reinforcement learning approach in which file copy time is used as feedback reward reinforcement, wherein
- the temporal difference learning, reinforcement learning approach uses a plurality of states respectively corresponding to a plurality of different numbers of threads to use to copy the files,
- the optimal number of threads for each respective discrete file size is determined by performing a plurality of training iterations, and
- in each training iteration, an action is taken to transition from a current state to a next state, and the files of each respective discrete file size are copied using a number of threads of the next state;
- and
- determining the continuous function based on the optimal number of threads to use for each respective discrete file size, by performing continuous function fitting on the optimal number of threads determined for each respective discrete file size, yielding the continuous function;
- after determining the continuous function, for a particular file to be copied that has the given file type and regardless of whether the particular file has a file size that is one of the plurality of discrete file sizes, inputting the file size of the particular file to the continuous function and receiving as output from the continuous function a number of threads to use to copy the particular file;
- and
- copying the particular file using the number of threads output by the continuous function.
A non-transitory computer-readable data storage medium storing program code executable by a processor to perform processing comprising:
- receiving a continuous function specific to a file type, the continuous function outputting a number of threads to use to copy an input file of any input file size that has the file type, the continuous function calculating the number of threads to use to copy the input file based on a file size of the input file, the continuous function determined by performing continuous function fitting on optimal numbers of threads used to copy files having the file type and of respective discrete file sizes that are each a specific different file size, the optimal numbers of threads determined without using the continuous function via a temporal difference learning, reinforcement learning approach in which file copy time is used as feedback reward reinforcement, wherein
- the temporal difference learning, reinforcement learning approach uses a plurality of states respectively corresponding to a plurality of different numbers of threads to use to copy the files,
- the optimal numbers of threads for each respective discrete file size is determined by performing a plurality of training iterations, and
- in each training iteration, an action is taken to transition from a current state to a next state, and the files of each respective discrete size are copied using a number of threads of the next state;
- after receiving the continuous function, for a particular file to be copied that has the file type and regardless of whether the particular file has a file size that is one of the respective discrete file sizes, inputting the file size of the particular file to the continuous function and receiving as output from the continuous function a number of threads to use to copy the particular file;
- and
- copying the particular file using the number of threads output by the continuous function.
A computing system comprising:
- a memory storing a Q-learning table of cumulative values for a plurality of state-action pairs, the Q-learning table corresponding to a given file type, each state-action pair comprising one of a plurality of states and one of a plurality of actions, the plurality of states corresponding to different numbers of threads, the plurality of actions comprising incrementing a number of threads, decrementing the number of threads, and maintaining the number of threads;
- and
- a processor configured to determine a continuous function that is specific to the given file type and that outputs a number of threads to use to copy an input file of any input file size that has the given file type, the continuous function calculating the number of threads to use to copy the input file based on a file size of the input file, wherein the continuous function is determined by:
- for each respective discrete file size of a plurality of discrete file sizes that are each a specific different file size, determining, without using the continuous function, an optimal number of threads to use to copy files of the respective discrete file size that have the given file type, by iteratively performing, until a stable state has been reached:
- probabilistically selecting an action to transition from a current state to a next state using the Q-learning table, the action probabilistically selected based on the action of the state-action pair including the current state for which the Q-learning table has a highest cumulative value;
- copying the files of the respective discrete file size using the number of threads of the next state and monitoring file transfer times to copy the files;
- updating, within the Q-learning table, a cumulative value for a state-action pair corresponding to the current state and the probabilistically selected action based on a reward value taking into account the monitored file transfer times;
- for each respective discrete file size, setting the optimal number of threads for the respective discrete file size to the number of threads corresponding to the stable state that has been reached for the respective discrete file size;
- and
- determine the continuous function based on the optimal number of threads to use for each respective discrete file size, by performing continuous function fitting on a set of data points corresponding to the given file type and that comprises the optimal numbers of threads determined for each respective discrete file size of the plurality of discrete file sizes, yielding the continuous function.
Dependencies are inferred from the claim text.