Key phrase spotting

A system comprising:

• data processing hardware;
• and
• memory hardware storing instructions that when executed cause the data processing hardware to perform operations comprising:
• receiving a textual input characterizing a key phrase;
• determining, from the textual input for the key phrase, using a key phrase encoder network of a key phrase spotting system, a key phrase encoding for a sequence of target sub-word units that represent the key phrase;
• for each corresponding frame of multiple frames representing an incoming audio signal, processing, by an acoustic encoder network of the key phrase spotting system, the corresponding frame to generate an encoder output that represents an acoustic encoding of the corresponding frame;
• determining, without performing speech recognition and without generating a transcription of the incoming audio signal, the incoming audio signal encodes an utterance of the key phrase based on the key phrase encoding and the encoder outputs generated for the multiple frames;
• and
• in response to determining that the incoming audio signal encodes the utterance of the key phrase, triggering an automatic speech recognition system to generate a transcription of other words following the key phrase that are encoded in the incoming audio signal, the automatic speech recognition system different than the key phrase spotting system.