Signal processing apparatus, training apparatus, and method

A signal processing method, comprising:

• receiving first acoustic data of a sound of an input sound source;
• receiving a voice quality converter parameter, wherein
• the voice quality converter parameter is trained based on a discriminator parameter, a speaker ID of a target sound source, and first training data of the sound of the input sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, second training data of a sound of the target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source,
• the target sound source is different from the input sound source,
• the discriminator parameter discriminates the input sound source of the first acoustic data,
• the first training data and the second training data are based on second acoustic data of a mixed sound,
• the mixed sound includes the sound of the input sound source and the sound of the target sound source, and
• the second acoustic data is different from parallel data and clean data;
• and
• converting the first acoustic data of the input sound source to third acoustic data of voice quality of the target sound source, wherein the conversion of the first acoustic data to the third acoustic data is based on the voice quality converter parameter.

A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a computer, cause the computer to execute operations, the operations comprising:

• receiving first acoustic data of a sound of an input sound source;
• receiving a voice quality converter parameter, wherein
• the voice quality converter parameter is trained based on a discriminator parameter, a speaker ID of a target sound source, and first training data of the sound of the input sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, second training data of a sound of the target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source,
• the target sound source is different from the input sound source,
• the discriminator parameter discriminates the input sound source of the first acoustic data,
• the first training data and the second training data are based on second acoustic data of a mixed sound,
• the mixed sound includes the sound of the input sound source and the sound of the target sound source, and
• the second acoustic data is different from parallel data and clean data;
• and
• converting the first acoustic data of the input sound source to third acoustic data of voice quality of the target sound source, wherein the conversion of the first acoustic data to the third acoustic data is based on the voice quality converter parameter.

A signal processing apparatus, comprising:

• a central processing apparatus configured to:
• receive specific acoustic data of a mixed sound, wherein
• the mixed sound includes a target sound of a target sound source and a non-target sound of a non-target sound source, and
• the target sound source is different from the non-target sound source;
• execute sound source separation to separate the specific acoustic data into first acoustic data of the target sound source and second acoustic data of the non-target sound source;
• receive a voice quality converter parameter, wherein
• the voice quality converter parameter is trained based on a discriminator parameter, a speaker ID of the target sound source, and first training data of a sound of an input sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, second training data of the target sound of the target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source,
• the target sound source is different from the input sound source,
• the discriminator parameter discriminates the target sound source of the first acoustic data,
• the first training data is based on the specific acoustic data of the mixed sound, and
• the second acoustic data is different from parallel data and clean data;
• execute voice quality conversion on the first acoustic data of the target sound to obtain third acoustic data, wherein
• the conversion of the first acoustic data is based on the voice quality converter parameter, and
• the first acoustic data is different from the parallel data and the clean data;
• and
• synthesize the third acoustic data and the second acoustic data of the non-target sound.

A signal processing method, comprising:

• receiving specific acoustic data of a mixed sound, wherein
• the mixed sound includes a target sound of a target sound source and a non-target sound of a non-target sound source, and
• the target sound source is different from the non-target sound source;
• executing sound source separation to separate the specific acoustic data into first acoustic data of the target sound source and second acoustic data of the non-target sound source;
• receiving a voice quality converter parameter, wherein
• the voice quality converter parameter is trained based on a discriminator parameter, a speaker ID of the target sound source, and first training data of a sound of an input sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, second training data of the target sound of the target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source,
• the target sound source is different from the input sound source,
• the discriminator parameter discriminates the target sound source of the first acoustic data,
• the first training data is based on the specific acoustic data of the mixed sound, and
• the second acoustic data is different from parallel data and clean data;
• executing voice quality conversion on the first acoustic data of the target sound to obtain third acoustic data, wherein
• the conversion of the first acoustic data is based on the voice quality converter parameter, and
• the first acoustic data is different from the parallel data and the clean data;
• and
• synthesizing the third acoustic data and the second acoustic data of the non-target sound.

A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a computer, cause the computer to execute operations, the operations comprising:

• receiving specific acoustic data of a mixed sound, wherein
• the mixed sound includes a target sound of a target sound source and a non-target sound of a non-target sound source, and
• the target sound source is different from the non-target sound source;
• executing sound source separation to separate the specific acoustic data into first acoustic data of the target sound source and second acoustic data of the non-target sound source;
• receiving a voice quality converter parameter, wherein
• the voice quality converter parameter is trained based on a discriminator parameter, a speaker ID of the target sound source, and first training data of a sound of an input sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, second training data of the target sound of the target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source,
• the target sound source is different from the input sound source,
• the discriminator parameter discriminates the target sound source of the first acoustic data,
• the first training data is based on the specific acoustic data of the mixed sound, and
• the second acoustic data is different from parallel data and clean data;
• executing voice quality conversion on the first acoustic data of the target sound to obtain third acoustic data, wherein
• the conversion of the first acoustic data is based on the voice quality converter parameter, and
• the first acoustic data is different from the parallel data and the clean data;
• and
• synthesizing the third acoustic data and the second acoustic data of the non-target sound.

A training apparatus, comprising:

• a central processing unit (CPU) configured to:
• receive first training data of a sound of an input sound source, second training data of a sound of a target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source, wherein
• the first training data and the second training data are based on acoustic data of a mixed sound,
• the acoustic data is different from parallel data and clean data,
• the mixed sound includes the sound of the input sound source and the sound of the target sound source, and
• the target sound source is different from the input sound source;
• train a discriminator parameter based on the first training data of the sound of the input sound source, the second training data of the sound of the target sound source, and the third training data of the sound of the sound source different from the input sound source and the target sound source,
• wherein the discriminator parameter is for discrimination of the input sound source;
• generate a voice quality converter parameter based on the first training data of the sound of the input sound source, the discriminator parameter, and a speaker ID of the target sound source;
• and
• output the generated voice quality converter parameter.

A training method, comprising:

• receiving first training data of a sound of an input sound source, second training data of a sound of a target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source, wherein
• the first training data and the second training data are based on acoustic data of a mixed sound,
• the acoustic data is different from parallel data and clean data,
• the mixed sound includes the sound of the input sound source and the sound of the target sound source, and
• the target sound source is different from the input sound source;
• training a discriminator parameter based on the first training data of the sound of the input sound source, the second training data of the sound of the target sound source, and the third training data of the sound of the sound source different from the input sound source and the target sound source,
• wherein the discriminator parameter is for discrimination of the input sound source;
• generating a voice quality converter parameter based on the first training data of the sound of the input sound source, the discriminator parameter, and a speaker ID of the target sound source;
• and
• outputting the generated voice quality converter parameter.

A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a computer, cause the computer to execute operations, the operations comprising:

• receiving first training data of a sound of an input sound source, second training data of a sound of a target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source, wherein
• the first training data and the second training data are based on acoustic data of a mixed sound,
• the acoustic data is different from parallel data and clean data,
• the mixed sound includes the sound of the input sound source and the sound of the target sound source, and
• the target sound source is different from the input sound source;
• training a discriminator parameter based on the first training data of the sound of the input sound source, the second training data of the sound of the target sound source, and the third training data of the sound of the sound source different from the input sound source and the target sound source,
• wherein the discriminator parameter is for discrimination of the input sound source;
• generating a voice quality converter parameter based on the first training data of the sound of the input sound source, the discriminator parameter, and a speaker ID of the target sound source;
• and
• outputting the generated voice quality converter parameter.

A training apparatus, comprising:

• a central processing unit (CPU) configured to:
• receive first training data of a sound of an input sound source, second training data of a sound of a target sound source, and a discriminator parameter, wherein
• the first training data and the second training data are based on a mixed sound including the sound of the input sound source and the sound of the target sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, the second training data of the sound of the target sound source, and third training data of a sound of a sound source different from the input sound source and the target sound source, and
• the input sound source is different from the target sound source;
• and
• train a voice quality converter parameter for conversion of first acoustic data of the sound of the input sound source to second acoustic data of voice quality of the target sound source, wherein
• the first acoustic data is different from parallel data and clean data,
• the voice quality converter parameter is trained based on the received first training data of the input sound source, a speaker ID of the target sound source, and the discriminator parameter, and
• the discriminator parameter discriminates the input sound source of the first acoustic data.

A training method, by a training apparatus, comprising:

• receiving first training data of a sound of an input sound source, second training data of a sound of a target sound source, and a discriminator parameter, wherein
• the first training data and the second training data are based on a mixed sound including the sound of the input sound source and the sound of the target sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, the second training data of the sound of the target sound source, and third training data of a sound source different from the input sound source and the target sound source, and
• the input sound source is different from the target sound source;
• and
• training a voice quality converter parameter for conversion of first acoustic data of the sound of the input sound source to second acoustic data of voice quality of the target sound source, wherein
• the first acoustic data is different from parallel data and clean data,
• the voice quality converter parameter is trained based on the received first training data of the input sound source, a speaker ID of the target sound source, and the discriminator parameter, and
• the discriminator parameter discriminates the input sound source of the first acoustic data.

A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a computer, cause the computer to execute operations, the operations comprising:

• receiving first training data of a sound of an input sound source, second training data of a sound of a target sound source, and a discriminator parameter, wherein
• the first training data and the second training data are based on a mixed sound including the sound of the input sound source and the sound of the target sound source,
• the discriminator parameter is trained based on the first training data of the sound of the input sound source, the second training data of the sound of the target sound source, and third training data of a sound source different from the input sound source and the target sound source, and
• the input sound source is different from the target sound source;
• and
• training a voice quality converter parameter for conversion of first acoustic data of the sound of the input sound source to second acoustic data of voice quality of the target sound source, wherein
• the first acoustic data is different from parallel data and clean data,
• the voice quality converter parameter is trained based on the received first training data of the input sound source, a speaker ID of the target sound source, and the discriminator parameter, and
• the discriminator parameter discriminates the input sound source of the first acoustic data.