End-to-end automatic speech recognition with transformer

A non-transitory computer storage that stores executable program instructions that, when executed by one or more computing devices, configure the one or more computing devices to perform operations comprising:

• generating a combination speech recognition pipeline by fusing a first speech recognition pipeline with a transformer, the transformer having an input layer, wherein the fusing comprises:
• configuring the terminating layer of the first pipeline to generate a learned output compatible with the input layer of the transformer;
• and
• training the combination speech recognition pipeline as a single model, to receive an audio clip and generate a transcript of the audio clip.