# US 2021/0280165 A1 · Unsupervised singing voice conversion with pitch adversarial network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0280165 A1
- Document ID: US20210280165A1
- Status: Published application
- Kind: A1
- Application number: 16/807,851
- Filed: 2020-03-03
- Published: 2021-09-09
- Earliest claimed: 2020-03-03
- Assignees: Tencent America LLC
- Applicants: Tencent America LLC
- Inventors: Chengzhu YU; Heng Lu; Chao Weng; Dong Yu
- Web page: https://patentgrep.com/patent/US20210280165A1

## CPC

- G10L 13/0335: {Pitch control}
- G10L 21/013: Adapting to target pitch
- G10L 13/047: Architecture of speech synthesisers
- G10L 25/90: Pitch determination of speech signals
- G10H 2210/066: for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
- G10H 2250/311: Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation
- G10H 2250/455: Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis
- G10L 25/30: using neural networks
- G10L 2021/0135: {Voice conversion or morphing}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE

## Abstract

A method, a computer readable medium, and a computer system are provided for singing voice conversion. Data corresponding to a singing voice is received. One or more features and pitch data are extracted from the received data using one or more adversarial neural networks. One or more audio samples are generated based on the extracted pitch data and the one or more features.

## Claims

### Claim 1

A method for singing voice conversion performed by one or more computer processors, comprising:
receiving data corresponding to a singing voice;
extracting one or more features from the received data;
extracting pitch data from the received data based on a pitch regression adversarial neural network including a dropout layer, two convolutional neural networks, and a fully connected layer; and
generating one or more audio samples based on the extracted pitch data and the one or more features.

### Claim at position 2

2-3. (canceled)

### Claim 4 (depends on 1)

The method of claim 1, wherein the features are extracted based on an identification of a singer associated with the singing voice.

### Claim 5 (depends on 4)

The method of claim 4, wherein the identification is performed by a singer classification adversarial neural network.

### Claim 6 (depends on 5)

The method of claim 5, wherein the singer classification adversarial neural network comprises a dropout layer, two convolutional neural networks, and a fully connected layer.

### Claim 7 (depends on 1)

The method of claim 1, further comprising calculating a singer classification loss value and a pitch regression loss value.

### Claim 8 (depends on 7)

The method of claim 7, wherein the singer classification loss value and pitch regression loss value are used as training values based on minimizing the singer classification loss value and pitch regression loss value.

### Claim 9 (depends on 1)

The method of claim 1, wherein the received singing voice data is compressed using an average pooling function.

### Claim 10 (depends on 1)

The method of claim 1, wherein the audio samples are generated without parallel data and without changing the content associated with the singing voice.

### Claim 11

A computer system for singing voice conversion, the computer system comprising:
one or more computer-readable non-transitory storage media configured to store computer program code; and
one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:
receiving code configured to cause the one or more computer processors to receive data corresponding to a singing voice;
first extracting code configured to cause the one or more computer processors to extract one or more features from the received data;
second extracting code configured to cause the one or more computer processors to extract pitch data from the received data based on a pitch regression adversarial neural network including a dropout layer, two convolutional neural networks, and a fully connected layer; and
generating code configured to cause the one or more computer processors to generate one or more audio samples based on the extracted pitch data and the one or more features.

### Claim at position 11

12-13. (canceled)

### Claim 14 (depends on 11)

The computer system of claim 11, wherein the features are extracted based on an identification of a singer associated with the singing voice.

### Claim 15 (depends on 14)

The computer system of claim 14, wherein the identification is performed by a singer classification adversarial neural network.

### Claim 16 (depends on 15)

The computer system of claim 15, wherein the singer classification adversarial neural network comprises a dropout layer, two convolutional neural networks, and a fully connected layer.

### Claim 17 (depends on 11)

The computer system of claim 11, further comprising calculating code configured to cause the one or more computer processors to calculate a singer classification loss value and a pitch regression loss value, wherein the singer classification loss value and pitch regression loss value are used as training values based on minimizing the singer classification loss value and pitch regression loss value.

### Claim 18 (depends on 11)

The computer system of claim 11, wherein the received singing voice data is compressed using an average pooling function.

### Claim 19 (depends on 11)

The computer system of claim 11, wherein the audio samples are generated without parallel data and without changing the content associated with the singing voice.

### Claim 20

A non-transitory computer readable medium having stored thereon a computer program for singing voice conversion, the computer program configured to cause one or more computer processors to:
receive data corresponding to a singing voice;
extract one or more features from the received data;
extract pitch data from the received data based on a pitch regression adversarial neural network including a dropout layer, two convolutional neural networks, and a fully connected layer; and
generate one or more audio samples based on the extracted pitch data and the one or more features.
