Learning apparatus, learning method, and recording medium
AssigneeNippon Telegraph and Telephone Corporation
InventorsYasutoshi IDA, Yasuhiro FUJIWARA
Life of this patent
Estimating the term…
Abstract
A learning apparatus according to the present invention is a learning apparatus that performs learning using a stochastic gradient descent method in machine learning, and includes: a processor configured to: calculate a first-order gradient in the stochastic gradient descent method; calculate a statistic of the first-order gradient; remove an initialization bias when calculating the statistic of the first-order gradient from the statistic of the first-order gradient calculated; adjust a learning rate by dividing the learning rate by standard deviation of the first-order gradient based on the statistic of the first-order gradient; and update a parameter of a learning model using the learning rate adjusted.
Claims7
A learning apparatus that performs learning using a stochastic gradient descent method in machine learning, the learning apparatus comprising:
- a processor configured to:
- calculate a first-order gradient in the stochastic gradient descent method;
- calculate a statistic of the first-order gradient;
- remove an initialization bias when calculating the statistic of the first-order gradient from the statistic of the first-order gradient calculated;
- adjust a learning rate by dividing the learning rate by standard deviation of the first-order gradient based on the statistic of the first-order gradient;
- and
- update a parameter of a learning model using the learning rate adjusted.
A learning method executed by a learning apparatus that performs learning using a stochastic gradient descent method in machine learning, the learning method comprising:
- calculating a first-order gradient in the stochastic gradient descent method;
- calculating a statistic of the first-order gradient;
- removing an initialization bias when calculating the statistic of the first-order gradient in calculation of the statistic from the statistic of the first-order gradient;
- adjusting a learning rate by dividing the learning rate by standard deviation of the first-order gradient based on the statistic of the first-order gradient, by a processor;
- and
- updating a parameter of a learning model using the learning rate adjusted in the adjustment.
A non-transitory computer-readable recording medium storing therein a learning program that causes a computer to execute a process comprising:
- calculating a first-order gradient in a stochastic gradient descent method in a case where learning is executed using the stochastic gradient descent method in machine learning;
- calculating a statistic of the first-order gradient;
- removing an initialization bias used in calculating the statistic of the first-order gradient at the calculating the statistic from the statistic of the first-order gradient;
- adjusting a learning rate by dividing the learning rate by standard deviation of the first-order gradient based on the statistic of the first-order gradient;
- and
- updating a parameter of a learning model using the learning rate adjusted at the adjusting.
Dependencies are inferred from the claim text.