US 11,599,798 B2 · Methods of operating a graphics processing unit (GPU) to train a deep neural network using a GPU local memory and related articles of manufacture

Abstract

A method operating a Graphics Processing Unit (GPU) memory can be provided by accessing specified training parameters used to train a Deep Neural Network (DNN) using a GPU with a local GPU memory, the specified training parameters including at least a specified batch size of samples configured to train the DNN. A sub-batch size of the samples can be defined that is less than or equal to the specified batch size of samples in response to determining that an available size of the local GPU memory is insufficient to store all data associated with training the DNN using one batch of the samples. Instructions configured to train the DNN using the sub-batch size can be defined so that an accuracy of the DNN trained using the sub-batch size is about equal to an accuracy of the DNN trained using the specified batch size of the samples.

Claims17

Dependent claims

A method operating a Graphics Processing Unit (GPU), the method comprising:

accessing specified training parameters used to train a Deep Neural Network (DNN) using a GPU with a local GPU memory, the specified training parameters including at least a specified batch size of samples configured to train the DNN;
defining a sub-batch size of the samples that is less than or equal to the specified batch size of samples in response to determining that an available size of the local GPU memory is insufficient to store all data associated with training the DNN using one batch of the samples;
and
generating instructions configured to train the DNN using the sub-batch size so that an accuracy of the DNN trained using the sub-batch size is equal to an accuracy of the DNN trained using the specified batch size of the samples, wherein the method further comprises:
prior to training the DNN, determining a static schedule of off-loading data to a host and data prefetching from the host for tasks to be used during training of the DNN based on a simulation of training the DNN;
and
applying the static schedule of off-loading data and data prefetching during the training of the DNN.

4 more elements

A non-transitory computer-readable medium whose contents, when executed by a computing system, cause the computing system to perform operations for operating a Graphics Processing Unit (GPU), the operations comprising:

accessing specified training parameters used to train a Deep Neural Network (DNN) using a GPU with a local GPU memory, the specified training parameters including at least a specified batch size of samples configured to train the DNN;
defining a sub-batch size of the samples that is less than or equal to the specified batch size of samples in response to determining that an available size of the local GPU memory is insufficient to store all data associated with training the DNN using one batch of the samples;
and
generating instructions configured to train the DNN using the sub-batch size so that an accuracy of the DNN trained using the sub-batch size is equal to an accuracy of the DNN trained using the specified batch size of the samples, the operations further comprising:
prior to training the DNN, determining a static schedule of off-loading data to a host and data prefetching from the host for tasks to be used during training of the DNN based on a simulation of training the DNN;
and
applying the static schedule of data off-loading and data prefetching during the training of the DNN.

4 more elements

accessing specified training parameters used to train a Deep Neural Network (DNN) using a GPU with a local GPU memory, the specified training parameters including at least a specified batch size of samples configured to train the DNN;
defining a sub-batch size of the samples that is less than or equal to the specified batch size of samples in response to determining that an available size of the local GPU memory is insufficient to store all data associated with training the DNN using one batch of the samples;
and
generating instructions configured to train the DNN using the sub-batch size so that an accuracy of the DNN trained using the sub-batch size is equal to an accuracy of the DNN trained using the specified batch size of the samples, wherein the sub-batch size of the samples is defined using the following relationship:
max
1
≤
t
≤
❘
"\[LeftBracketingBar]"
𝕋
❘
"\[RightBracketingBar]"
-
α
⁢
❘
"\[LeftBracketingBar]"
𝕋
❘
"\[RightBracketingBar]"
+
1
{
∑
k
=
t
t
+
α
⁢
❘
"\[LeftBracketingBar]"
𝕋
❘
"\[RightBracketingBar]"
-
1
[
∑
d
⁡
(
b
)
∈
𝕀
⁡
(
k
,
b
)
Size
(
d
⁡
(
b
)
)
+
Size
(
O
⁡
(
k
,
b
)
)
]
-
Size
(
reused
⁢
data
,
b
)
+
max
t
≤
k
≤
t
+
α
⁢
❘
"\[LeftBracketingBar]"
𝕋
❘
"\[RightBracketingBar]"
-
1
{
WS
⁡
(
α
fastest
(
k
)
,
b
)
}
}
+
S
W
,
wherein T is all consecutive tasks occurring in topographical order in a task flow data graph representing training of the DNN, the first term in the outer max operator is the sum of input data and output data for 15% of consecutive tasks occurring in topographical order in T, the second term in the outer max operator is the total size of reused data in the 15% of consecutive tasks occurring in topographical order in T, and the third term is a maximum work space size for the fastest convolution kernel to complete each task in the 15% of consecutive tasks occurring in topographical order in T.

117 more elements

Dependencies are inferred from the claim text.

Methods of operating a graphics processing unit (GPU) to train a deep neural network using a GPU local memory and related articles of manufacture

Life of this patent

Methods of operating a graphics processing unit (GPU) to train a deep neural network using a GPU local memory and related articles of manufacture

Life of this patent

Abstract

Claims17