# US 2025/0095646 A1 · Automatic replacement of targeted objects within arbitrary media > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0095646 A1 - Document ID: US20250095646A1 - Status: Published application - Kind: A1 - Application number: 18/470,061 - Filed: 2023-09-19 - Published: 2025-03-20 - Earliest claimed: 2023-09-19 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Hessel Tuinhof; Marco Simioni; Killian Levacher; Stefano Braghin - Web page: https://patentgrep.com/patent/US20250095646A1 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 21/00: Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence) - G06N 3/047: Probabilistic or stochastic networks - G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser - G10L 15/08: Speech classification or search - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 13/02: Methods for producing synthetic speech; Speech synthesisers - G10L 25/30: using neural networks - G10L 2015/088: {Word spotting} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract In an approach to improve the protection of sensitive data, embodiments extract, by a signal separator component, a voice from an audio file, and transcribe, by a speech-to-text component, the voice in the audio file into text. Further, embodiments identify, by a natural language processing and identification component, sensitive data in the text, wherein identifying sensitive date comprises parsing the text and identifying the sensitive data contained within the text. Additionally, embodiments identify, by a voice locator component, a synthetic voice that matches the voice from the audio file, replace, by a voice replacer component, the identified sensitive data with synthetic data that is semantically and contextually meaningful, wherein the synthetic data is voiced using the synthetic voice, and output a new audio file with the sensitive data replaced by the synthetic data. ## Claims ### Claim 1 A computer-implemented method comprising: extracting, by a signal separator component, a voice from an audio file; transcribing, by a speech-to-text component, the voice in the audio file into text; identifying, by a natural language processing and identification component, sensitive data in the text, wherein identifying sensitive date comprises: parsing the text and identifying the sensitive data contained within the text; identifying, by a voice locator component, a synthetic voice that matches the voice from the audio file; replacing, by a voice replacer component, the identified sensitive data with synthetic data that is semantically and contextually meaningful, wherein the synthetic data is voiced using the synthetic voice; and outputting a new audio file with the sensitive data replaced by the synthetic data. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: receiving an audio file with editable speech content, wherein audio file editing utilizes an input query, configuration, and a knowledge base. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising: tagging and categorizing an original user's speech signal, transcript, and/or sensitive words; and marking the tagged and categorized sensitive words in the text with a timestamp associated with a location of the sensitive words in the audio file. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising: identifying, within a manifold of a pre-trained generative adversarial network (GAN) that synthetizes the voice, the synthetic voice that matches the voice from the audio file, wherein the synthetic voice that matches the voice from the audio file is determined based on predetermined metrics. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: dynamically tunning, by a voice tuner component, the synthetic voice, wherein the synthetic voice is dynamically tuned until the synthetic voice is within a predetermined threshold of acceptance of similarity associated with the voice from the audio file. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising: utilizing the sensitive data from the text to identify replacements for the sensitive data within a knowledge base; and outputting a list of replacement options for the identified sensitive data. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising: merging the new audio file with the audio file to generate an updated audio file; and outputting the updated audio file, wherein the updated audio file comprises audio with the sensitive data replaced with synthetic data voice by the synthetic voice. ### Claim 8 A computer system comprising: one or more computer processors; one or more computer readable storage devices; program instructions stored on the one or more computer readable storage devices for execution by at least one of the one or more computer processors, the stored program instructions comprising: program instructions to extract, by a signal separator component, a voice from an audio file; program instructions to transcribe, by a speech-to-text component, the voice in the audio file into text; program instructions to identify, by a natural language processing and identification component, sensitive data in the text, wherein identifying sensitive date comprises: program instructions to parse the text and identifying the sensitive data contained within the text; program instructions to identify, by a voice locator component, a synthetic voice that matches the voice from the audio file; program instructions to replace, by a voice replacer component, the identified sensitive data with synthetic data that is semantically and contextually meaningful, wherein the synthetic data is voiced using the synthetic voice; and program instructions to output a new audio file with the sensitive data replaced by the synthetic data. ### Claim 9 (depends on 8) The computer system of claim 8, further comprising: program instructions to receive an audio file with editable speech content, wherein audio file editing utilizes an input query, configuration, and a knowledge base. ### Claim 10 (depends on 8) The computer system of claim 8, further comprising: program instructions to tag and categorize an original user's speech signal, transcript, and/or sensitive words; and program instructions to mark the tagged and categorized sensitive words in the text with a timestamp associated with a location of the sensitive words in the audio file. ### Claim 11 (depends on 8) The computer system of claim 8, further comprising: program instructions to identify, within a manifold of a pre-trained generative adversarial network (GAN) that synthetizes the voice, the synthetic voice that matches the voice from the audio file, wherein the synthetic voice that matches the voice from the audio file is determined based on predetermined metrics. ### Claim 12 (depends on 8) The computer system of claim 8, further comprising: program instructions to dynamically tune, by a voice tuner component, the synthetic voice, wherein the synthetic voice is dynamically tuned until the synthetic voice is within a predetermined threshold of acceptance of similarity associated with the voice from the audio file. ### Claim 13 (depends on 8) The computer system of claim 8, further comprising: program instructions to utilize the sensitive data from the text to identify replacements for the sensitive data within a knowledge base; and program instructions to output a list of replacement options for the identified sensitive data. ### Claim 14 (depends on 8) The computer system of claim 8, further comprising: program instructions to merge the new audio file with the audio file to generate an updated audio file; and program instructions to output the updated audio file, wherein the updated audio file comprises audio with the sensitive data replaced with synthetic data voice by the synthetic voice. ### Claim 15 A computer program product comprising: one or more computer readable storage devices and program instructions stored on the one or more computer readable storage devices, the stored program instructions comprising: program instructions to extract, by a signal separator component, a voice from an audio file; program instructions to transcribe, by a speech-to-text component, the voice in the audio file into text; program instructions to identify, by a natural language processing and identification component, sensitive data in the text, wherein identifying sensitive date comprises: program instructions to parse the text and identifying the sensitive data contained within the text; program instructions to identify, by a voice locator component, a synthetic voice that matches the voice from the audio file; program instructions to replace, by a voice replacer component, the identified sensitive data with synthetic data that is semantically and contextually meaningful, wherein the synthetic data is voiced using the synthetic voice; and program instructions to output a new audio file with the sensitive data replaced by the synthetic data. ### Claim 16 (depends on 15) The computer program product of claim 15, further comprising: program instructions to receive an audio file with editable speech content, wherein audio file editing utilizes an input query, configuration, and a knowledge base. ### Claim 17 (depends on 15) The computer program product of claim 15, further comprising: program instructions to tag and categorize an original user's speech signal, transcript, and/or sensitive words; and program instructions to mark the tagged and categorized sensitive words in the text with a timestamp associated with a location of the sensitive words in the audio file. ### Claim 18 (depends on 15) The computer program product of claim 15, further comprising: program instructions to identify, within a manifold of a pre-trained generative adversarial network (GAN) that synthetizes the voice, the synthetic voice that matches the voice from the audio file, wherein the synthetic voice that matches the voice from the audio file is determined based on predetermined metrics. ### Claim 19 (depends on 15) The computer program product of claim 15, further comprising: program instructions to dynamically tune, by a voice tuner component, the synthetic voice, wherein the synthetic voice is dynamically tuned until the synthetic voice is within a predetermined threshold of acceptance of similarity associated with the voice from the audio file. ### Claim 20 (depends on 15) The computer program product of claim 15, further comprising: program instructions to utilize the sensitive data from the text to identify replacements for the sensitive data within a knowledge base; program instructions to output a list of replacement options for the identified sensitive data; program instructions to merge the new audio file with the audio file to generate an updated audio file; and program instructions to output the updated audio file, wherein the updated audio file comprises audio with the sensitive data replaced with synthetic data voice by the synthetic voice.