System and method for singing synthesis

A singing synthesis system comprising:

• a recording section operable to record a plurality of vocals when a singer sings a part or entirety of a song a plurality of times;
• an estimation and analysis data storing section operable to:
• estimate time periods of a plurality of phonemes in a phoneme unit for the respective vocals sung by the singer the plurality of times that have been recorded by the recording section and store the estimated time periods;
• and
• obtain pitch data, power data, and timbre data by analyzing a pitch, a power, and a timbre of each vocal and store the obtained pitch data, the obtained power data, and the obtained timbre data;
• an estimation and analysis results display section operable to display on a display screen reflected pitch data, reflected power data, and reflected timbre data, whereby estimation and analysis results have been reflected in the pitch data, the power data, and the timbre data, together with the time periods of the plurality of phonemes recorded in the estimation and analysis data storing section;
• a data selecting section configured to allow a user to select the pitch data, the power data, and the timbre data for the respective time periods of the phonemes from the estimation and analysis results for the respective vocals sung by the singer the plurality of times as displayed on the display screen;
• an integrated singing data generating section operable to generate integrated singing data by integrating the pitch data, the power data, and the timbre data, which have been selected by using the data selecting section, for the respective time periods of the phonemes;
• and
• a singing playback section operable to play back the integrated singing data.

A singing synthesis method comprising:

• a data storing step of storing in a data storage section a music audio signal and lyrics data temporally aligned with the music audio signal;
• a display step of displaying on a display screen of a display section at least a part of lyrics, based on the lyrics data;
• a playback step of playing back in a music audio signal playback section the music audio signal from a signal portion or its immediately preceding signal portion of the music audio signal corresponding to a character in the lyrics when the character in the lyrics displayed on the display screen is selected due to a selection operation;
• a recording step of recording in a recording section a plurality of vocals sung by a singer a plurality of times, listening to played-back music while the music audio signal playback section plays back the music audio signal;
• an estimation and analysis data storing step of estimating time periods of a plurality of phonemes in a phoneme unit for the respective vocals sung by the singer the plurality of times that have been recorded in the recording section and storing the estimated time periods in an estimation and analysis data storing section;
• and obtaining pitch data, power data, and timbre data by analyzing a pitch, a power, and a timbre of each vocal, and storing the obtained pitch, the obtained power and the obtained timbre data in the estimation and analysis data storing section;
• an estimation and analysis results displaying step of displaying on the display screen reflected pitch data, reflected power data, and reflected timbre data, whereby estimation and analysis results have been reflected in the pitch data, the power data, and the timbre data, together with the time periods of the plurality of phonemes recorded in the estimation and analysis data storing section;
• a data selecting step of allowing a user to select, by using a data selecting section, the pitch data, the power data, and the timbre data for the respective time periods of the phonemes from the estimation results for the respective vocals sung by the singer the plurality of times as displayed on the display screen;
• an integrated singing data generating step of generating integrated singing data by integrating the pitch data, the power data, and the timbre data, which have been selected by using the data selecting section, for the respective time periods of the phonemes;
• and
• a singing playback step of playing back the integrated singing data.

A singing synthesis method comprising:

• a recording step of recording a plurality of vocals when a singer sings a part or entirety of a song a plurality of times;
• an estimation and analysis data storing step of estimating time periods of a plurality of phonemes in a phoneme unit for the respective vocals sung by the singer the plurality of times that have been recorded by the recording step, and storing the estimated time periods in an estimation and analysis data storing section;
• and obtaining pitch data, power data, and timbre data by analyzing a pitch, a power, and a timbre of each vocal, and storing the obtained pitch, the obtained power and the obtained timbre data in the estimation and analysis data storing section;
• an estimation and analysis results displaying step of displaying on a display screen reflected pitch data, reflected power data, and reflected timbre data, whereby estimation and analysis results have been reflected in the pitch data, the power data, and the timbre data, together with the time periods of the plurality of phonemes recorded in the estimation and analysis data storing section;
• a data selecting step of allowing a user to select, by using a data selecting section, the pitch data, the power data, and the timbre data for the respective time periods of the phonemes from the estimation results for the respective vocals sung by the singer the plurality of times as displayed on the display screen;
• an integrated singing data generating step of generating integrated singing data by integrating the pitch data, the power data, and the timbre data, which have been selected by the data selecting step, for the respective time periods of the phonemes;
• and
• a singing playback step of playing back the integrated singing data.