Electronic musical instrument, electronic musical instrument control method, and storage medium

An electronic musical instrument comprising:

• a plurality of operation elements respectively corresponding to mutually different pitch data;
• a memory that stores a trained acoustic model obtained by performing machine learning on training musical score data including training lyric data and training pitch data, and on training singing voice data of a singer corresponding to the training musical score data, the trained acoustic model being configured to receive lyric data and pitch data and output acoustic feature data of a singing voice of the singer in response to the received lyric data and pitch data;
• and
• at least one processor,
• wherein the at least one processor:
• in accordance with a user operation on an operation element in the plurality of operation elements, inputs prescribed lyric data and pitch data corresponding to the user operation of the operation element to the trained acoustic model so as to cause the trained acoustic model to output the acoustic feature data in response to the inputted prescribed lyric data and the inputted pitch data, and
• digitally synthesizes and outputs inferred singing voice data that infers a singing voice of the singer on the basis of the acoustic feature data output by the trained acoustic model in response to the inputted prescribed lyric data and the inputted pitch data,
• wherein the plurality of operation elements include a first operation element as the operation element that was operated by the user and a second operation element that meets a prescribed condition with respect to the first operation element, and
• wherein the at least one processor applies an acoustic effect to the inferred singing voice data when the second operation element is operated while the first operation element is being operated.

A method performed by at least one processor in an electronic musical instrument that includes, in addition to the at least one processor:

• a plurality of operation elements respectively corresponding to mutually different pitch data;
• and a memory that stores a trained acoustic model obtained by performing machine learning on training musical score data including training lyric data and training pitch data, and on training singing voice data of a singer corresponding to the training musical score data, the trained acoustic model being configured to receive lyric data and prescribed pitch data and output acoustic feature data of a singing voice of the singer, the method comprising, via the at least one processor, the following:
• in accordance with a user operation on an operation element in the plurality of operation elements, inputting prescribed lyric data and pitch data corresponding to the user operation of the operation element to the trained acoustic model so as to cause the trained acoustic model to output the acoustic feature data in response to the inputted prescribed lyric data and the inputted pitch data, and
• digitally synthesizing and outputting inferred singing voice data that infers a singing voice of the singer on the basis of the acoustic feature data output by the trained acoustic model in response to the inputted prescribed lyric data and the inputted pitch data,
• wherein the memory contains melody pitch data indicating operation elements that a user is to operate, singing voice output timing data indicating output timings at which respective singing voices for pitches indicated by the melody pitch data are to be output, and lyric data respectively corresponding to the melody pitch data, and
• wherein the method includes via said at least one processor:
• when a user operation for producing a singing voice is performed at an output timing indicated by the singing voice output timing data, inputting pitch data corresponding to the user-operated operation element and lyric data corresponding to said output timing to the trained acoustic model, and outputting, at said output timing, inferred singing voice data that infers the singing voice of the singer on the basis of the acoustic feature data output by the trained acoustic model in response to the input, and
• when a user operation for producing a singing voice is not performed at the output timing indicated by the singing voice output timing data, inputting melody pitch data corresponding to said output timing and lyric data corresponding to said output timing to the trained acoustic model, and outputting, at said output timing, inferred singing voice data that infers the singing voice of the singer on the basis of the acoustic feature data output by the trained acoustic model in response to the input.

A method performed by at least one processor in an electronic musical instrument that includes, in addition to the at least one processor:

• a plurality of operation elements respectively corresponding to mutually different pitch data;
• and a memory that stores a trained acoustic model obtained by performing machine learning on training musical score data including training lyric data and training pitch data, and on training singing voice data of a singer corresponding to the training musical score data, the trained acoustic model being configured to receive lyric data and prescribed pitch data and output acoustic feature data of a singing voice of the singer, the method comprising, via the at least one processor, the following:
• in accordance with a user operation on an operation element in the plurality of operation elements, inputting prescribed lyric data and pitch data corresponding to the user operation of the operation element to the trained acoustic model so as to cause the trained acoustic model to output the acoustic feature data in response to the inputted prescribed lyric data and the inputted pitch data, and
• digitally synthesizing and outputting inferred singing voice data that infers a singing voice of the singer on the basis of the acoustic feature data output by the trained acoustic model in response to the inputted prescribed lyric data and the inputted pitch data,
• wherein the plurality of operation elements include a first operation element as the operation element that was operated by the user and a second operation element that meets a prescribed condition with respect to the first operation element, and
• wherein the method further includes, via the at least one processor, applying an acoustic effect to the inferred singing voice data when the second operation element is operated while the first operation element is being operated.