Adversarial speech-text protection against automated analysis
AssigneeInternational Business Machines Corporation
InventorsNgoc Minh Tran, Marco Simioni, Hessel Tuinhof
Life of this patent
Estimating the term…
Abstract
A method, system, and computer program product are disclosed. The method includes processing an audio signal that includes speech data and transcribing the speech data to generate text data. The method also includes identifying a vulnerable portion of the text data and, in response, applying adversarial text to the text data to generate robust text data. Adversarial noise corresponding to the robust text data is generated and applied to the speech data.
Claims20
A method, comprising:
- processing an audio signal comprising speech data;
- transcribing the speech data to generate text data;
- identifying a vulnerable portion of the text data;
- in response to the identifying, modifying the text data to generate a robust transcript, wherein the modifying comprises replacing the vulnerable portion of the text data with adversarial text;
- designing adversarial noise corresponding to the adversarial text;
- and
- applying the corresponding adversarial noise to the audio signal to generate a robust audio signal comprising modified speech data that, when transcribed, generates a transcript with a similarity to the robust transcript that is above a threshold similarity.
A system, comprising:
- a memory;
- and
- a processor communicatively coupled to the memory, wherein the processor is configured to perform a method comprising:
- processing an audio signal comprising speech data;
- transcribing the speech data to generate text data;
- identifying a vulnerable portion of the text data;
- in response to the identifying, modifying the text data to generate a robust transcript, wherein the modifying comprises replacing the vulnerable portion of the text data with adversarial text;
- designing adversarial noise corresponding to the adversarial text;
- and
- applying the corresponding adversarial noise to the audio signal to generate a robust audio signal comprising modified speech data that, when transcribed, generates a transcript with a similarity to the robust transcript that is above a threshold similarity.
A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause a device to perform a method, the method comprising:
- processing an audio signal comprising speech data;
- transcribing the speech data to generate text data;
- identifying a vulnerable portion of the text data;
- in response to the identifying, modifying the text data to generate a robust transcript, wherein the modifying comprises replacing the vulnerable portion of the text data with adversarial text;
- designing adversarial noise corresponding to the adversarial text;
- and
- applying the corresponding adversarial noise to the audio signal to generate a robust audio signal comprising modified speech data that, when transcribed, generates a transcript with a similarity to the robust transcript that is above a threshold similarity.
Dependencies are inferred from the claim text.