End-to-end streaming keyword spotting

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• obtaining a plurality of first training samples comprising utterances that contain a trigger phrase, each first training sample comprising:
• a respective sequence of input frames that each contain respective audio features characterizing phonetic components of the trigger phrase;
• and
• respective labels each assigned to a respective input frame in the respective sequence of input frames, each label indicating whether the respective audio features of the respective input frame assigned to the label characterize a phoneme in the trigger phrase or silence;
• obtaining a plurality of second training samples comprising additional utterances that do not contain the trigger phrase;
• and
• training, using cross-entropy loss, a neural network model on the plurality of first training samples and the plurality of second training samples to teach the neural network model to learn how to detect a presence of the trigger phrase in streaming audio.