Statistical-analysis-based reset of recurrent neural networks for automatic speech recognition

A system for calculating recurrent neural network (RNN) reset parameters, the system comprising:

• a statistics calculation circuit to calculate a statistic based on outputs of an RNN that is periodically re-initialized at a selected RNN reset time, the calculated statistic associated with the selected RNN reset time, and to generate a statistical sequence from a plurality of the calculated statistics wherein each of the plurality of calculated statistics is associated with a unique selected RNN reset time selected from a pre-determined range of RNN reset times;
• a statistics analysis circuit to analyze the sequence to identify a maximum interval during which variations of the sequence do not exceed a threshold value;
• and
• a reset-parameter calculation circuit to select a reset time parameter for re-initialization of the RNN during operation, the reset time parameter based on the identified maximum interval, select a reset context duration parameter for the re-initialization of the RNN during operation, the reset context duration parameter based on a time associated with a starting point of the identified maximum interval, and store the reset time parameter and the reset context duration parameter for use by a reset circuit to re-initialize the RNN during operation;
• wherein the RNN is an acoustic model for an automatic speech recognition system.

At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for calculating recurrent neural network (RNN) reset parameters, the operations comprising:

• calculating a statistic based on outputs of an RNN that is periodically re-initialized at a selected RNN reset time, the calculated statistic associated with the selected RNN reset time;
• generating a statistical sequence from a plurality of the calculated statistics wherein each of the plurality of calculated statistics is associated with a unique selected RNN reset time selected from a pre-determined range of RNN reset times;
• analyzing the sequence to identify a maximum interval during which variations of the sequence do not exceed a threshold value;
• selecting a reset time parameter for re-initialization of the RNN during operation, the reset time parameter based on the identified maximum interval;
• selecting a reset context duration parameter for the re-initialization of the RNN during operation, the reset context duration parameter based on a time associated with a starting point of the identified maximum interval;
• and
• storing the reset time parameter and the reset context duration parameter for use by a reset circuit to re-initialize the RNN during operation;
• wherein the RNN is an acoustic model for an automatic speech recognition system.