Multi-factor audio watermarking

A computer program product comprising one or more non-transitory computer-readable storage media having program instructions collectively stored on the one or more non-transitory computer-readable storage media, the program instructions executable to:

• receive, via one or more microphones of a client device, first audio data that captures a first spoken utterance;
• process the first audio data using automatic speech recognition to generate a speech transcription feature or intermediate embedding;
• detect a particular watermark that is embedded in the first audio data;
• in response to detecting the particular watermark, determine that the speech transcription feature or intermediate embedding corresponds to one of a plurality of stored speech transcription features or intermediate embeddings, and determine that the one of the plurality of stored speech transcription features or intermediate embeddings is associated with the particular watermark;
• and
• based on (i) detecting the particular watermark, (ii) the speech transcription feature or intermediate embedding corresponding to the one of the plurality of stored speech transcription features or intermediate embeddings, and (iii) the one of the plurality of stored speech transcription features or intermediate embeddings being associated with the particular watermark, modify a threshold that is indicative of one or more hotwords being present in audio data.

A system comprising:

• a processor, a computer-readable memory, one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to:
• receive, via one or more microphones of a client device, audio data that captures a spoken utterance;
• process the audio data using one or more machine learning models to generate a predicted output that indicates a probability of one or more hotwords being present in the audio data;
• determine that the predicted output satisfies a threshold that is indicative of the one or more hotwords being present in the audio data;
• in response to determining that the predicted output satisfies the threshold, process the audio data using automatic speech recognition to generate a speech transcription feature or intermediate embedding;
• detect a particular watermark that is embedded in the audio data;
• in response to detecting the particular watermark determine that the speech transcription feature or intermediate embedding corresponds to one of a plurality of stored speech transcription features or intermediate embeddings, and determine that the one of the plurality of stored speech transcription features or intermediate embeddings is associated with the particular watermark;
• and
• based on (i) detecting the particular watermark, (ii) the speech transcription feature or intermediate embedding corresponding to one of the plurality of stored speech transcription features or intermediate embeddings, and (iii) the one of the plurality of stored speech transcription features or intermediate embeddings being associated with the particular watermark, suppress processing of a query included in the audio data.