Speech recognition device, speech recognition method, and non-transitory computer-readable medium

A speech recognition method, comprising:

• by an acquisition part, acquiring a speech signal;
• by a speech feature amount calculation part, calculating a speech feature amount of the acquired speech signal;
• by a first speech recognition part, based on the speech feature amount, performing speech recognition using a first end-to-end model that has been learned, attaching a first tag to a vocabulary portion of a specific class in a text that is a recognition result, and outputting the vocabulary portion with the first tag;
• by a second speech recognition part, based on the speech feature amount, performing speech recognition using a second end-to-end model that has been learned, attaching a second tag to a vocabulary portion of a specific class in a phoneme that is a recognition result, and outputting the phoneme with the second tag;
• by a phoneme replacement part, replacing the vocabulary portion with the first tag in the text recognized by the first speech recognition part with a phoneme with the second tag;
• and
• by an output part, converting the phoneme with the second tag obtained by replacement by the phoneme replacement part into a text and outputting the converted text, wherein,
• in response to a text with a highest similarity in a language model in which text and phonemes are associated has the similarity greater than a threshold, the output part converts and outputs the phoneme with the second tag obtained by replacement by the phoneme replacement part;
• and
• in response to the text with highest similarity in a language model stored in a language model storage part has the similarity equal to or less than a threshold, the output part outputs the phoneme with the second tag obtained by replacement by the phoneme replacement part as the text with the first tag recognized by the first speech recognition part as it is.

A non-transitory computer-readable medium storing a program, the program causing a computer to:

• acquire a speech signal;
• calculate a speech feature amount of the acquired speech signal;
• based on the speech feature amount, perform speech recognition using a first end-to-end model that has been learned, attach a first tag to a vocabulary portion of a specific class in a text that is a recognition result, and output the vocabulary portion with the first tag;
• based on the speech feature amount, perform speech recognition using a second end-to-end model that has been learned, attach a second tag to a vocabulary portion of a specific class in a phoneme that is a recognition result, and output the phoneme with the second tag;
• replace the vocabulary portion with the first tag in the text recognized using the first end-to-end model with the phoneme with the second tag;
• and
• convert the phoneme with the second tag obtained by replacement into a text and output the converted text, wherein,
• in response to a text with a highest similarity in a language model in which text and phonemes are associated has the similarity greater than a threshold, the computer converts and outputs the phoneme with the second tag obtained by replacement by the phoneme replacement part;
• and
• in response to the text with highest similarity in a language model stored in a language model storage part has the similarity equal to or less than a threshold, the computer outputs the phoneme with the second tag obtained by replacement by the phoneme replacement part as the text with the first tag recognized by the first speech recognition part as it is.