Systems and methods for normalization in deep learning
AssigneeNTT Research, Inc.
InventorsHidenori TANAKA, Ekdeep Singh Lubana
Life of this patent
Estimating the term…
Abstract
A method of training a neural network may be provided. The method may include receiving a raw data set and normalizing the raw data set using a group normalization layer to generate a training data set for training a neural network, wherein the group normalization layer segments the raw data set into a plurality of groups of a predetermined size. The method may also include randomly initializing the neural network by randomly assigning values to a plurality of weights corresponding to a plurality of layers of the neural network. The method may further include training the neural network using the training data set.
Claims20
A method of training a neural network, the method comprising:
- receiving a raw data set;
- normalizing the raw data set using a group normalization layer to generate a training data set for training a neural network, wherein the group normalization layer segments the raw data set into a plurality of groups of a predetermined size;
- randomly initializing the neural network by randomly assigning values to a plurality of weights corresponding to a plurality of layers of the neural network;
- and
- training the neural network using the training data set.
A system comprising:
- a non-transitory storage medium storing computer program instructions;
- and
- one or more processors configured to execute the computer program instructions to cause operations comprising:
- receiving a raw data set;
- normalizing the raw data set using a group normalization layer to generate a training data set for training a neural network, wherein the group normalization layer segments the raw data set into a plurality of groups of a predetermined size;
- randomly initializing the neural network by randomly assigning values to a plurality of weights corresponding to a plurality of layers of the neural network;
- and
- training the neural network using the training data set.
Dependencies are inferred from the claim text.