Accuracy of streaming rnn transducer
AssigneeInternational Business Machines Corporation
InventorsGakuto Kurata, George Andrei Saon
Life of this patent
Estimating the term…
Abstract
A computer-implemented method is provided for model training. The method includes training a second end-to-end neural speech recognition model that has a bidirectional encoder to output same symbols from an output probability lattice of the second end-to-end neural speech recognition model as from an output probability lattice of a trained first end-to-end neural speech recognition model having a unidirectional encoder. The method also includes building a third end-to-end neural speech recognition model that has a unidirectional encoder by training the third end-to-end neural speech recognition model as a student by using the trained second end-to-end neural speech recognition model as a teacher in a knowledge distillation method.
Claims20
A computer-implemented method for model training, comprising:
- training a second end-to-end neural speech recognition model that has a bidirectional encoder to output same symbols from an output probability lattice of the second end-to-end neural speech recognition model as from an output probability lattice of a trained first end-to-end neural speech recognition model having a unidirectional encoder;
- and
- building a third end-to-end neural speech recognition model that has a unidirectional encoder by training the third end-to-end speech recognition model as a student by using the trained second end-to-end neural speech recognition model as a teacher in a knowledge distillation method.
A computer program product for model training, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
- training a second end-to-end neural speech recognition model that has a bidirectional encoder to output same symbols from an output probability lattice of the second end-to-end neural speech recognition model as from an output probability lattice of a trained first end-to-end neural speech recognition model having a unidirectional encoder;
- and
- building a third end-to-end neural speech recognition model that has a unidirectional encoder by training the third end-to-end speech recognition model as a student by using the trained second end-to-end neural speech recognition model as a teacher in a knowledge distillation method.
A computer processing system for model training, comprising:
- a memory device for storing program code;
- and
- a hardware processor operatively coupled to the memory device for running the program code to
- train a second end-to-end neural speech recognition model that has a bidirectional encoder to output same symbols from an output probability lattice of the second end-to-end neural speech recognition model as from an output probability lattice of a trained first end-to-end neural speech recognition model having a unidirectional encoder;
- and
- build a third end-to-end neural speech recognition model that has a unidirectional encoder by training the third end-to-end speech recognition model as a student by using the trained second end-to-end neural speech recognition model as a teacher in a knowledge distillation method.
Dependencies are inferred from the claim text.