Speech recognition method and apparatus

A neural network model training method, comprising:

• constructing a training sample, the training sample comprising a sample statement comprising a character, and a phoneme and a punctuation that correspond to the character in the sample statement;
• and
• training a second multitask neural network model using the training sample to obtain a first multitask neural network model, each of the second multitask neural network model and the first multitask neural network model being configured to output a first prediction result and to display at least a part of the first prediction result, the first prediction result comprising a character prediction result and a punctuation prediction result;
• wherein the training a second multitask neural network model comprises:
• inputting the training sample into the second multitask neural network model, determining a character probability matrix and a punctuation probability matrix that correspond to the training sample;
• separately calculating a character cross entropy loss and a punctuation cross entropy loss based on the character probability matrix and the punctuation probability matrix;
• calculating a weighted cross entropy loss based on the character cross entropy loss, a first weight corresponding to the character cross entropy loss, the punctuation cross entropy loss, and a second weight corresponding to the punctuation cross entropy loss;
• and
• adjusting a parameter of the second multitask neural network model based on the weighted cross entropy loss to obtain the trained first multitask neural network model.

A speech recognition apparatus, comprising:

• an input module configured to input a to-be-recognized phoneme into a first multitask neural network model stored in one or more memories of a data processing system;
• an inference module configured to automatically output a first prediction result based on execution by the data processing system of program instructions of the first multitask neural network model stored in the one or more memories of the data processing system and applied to the to-be-recognized phoneme, the first prediction result comprising a character prediction result and a punctuation prediction result each generated by the first multitask neural network model as applied to the to-be-recognized phoneme and corresponding to the to-be-recognized phoneme;
• and
• a display module configured to display at least a part of the first prediction result on a display of a terminal device based on the first prediction result;
• wherein:
• the first multitask neural network model is a streaming network structure;
• and
• the inputting, by a terminal device, a to-be-recognized phoneme into a first multitask neural network model, and outputting a first prediction result using the first multitask neural network model comprises:
• cyclically inputting, by the terminal device, the to-be-recognized phoneme into the first multitask neural network model, and outputting the first prediction result based on a length of a currently input to-be-recognized phoneme by using the first multitask neural network model;
• or
• the first multitask neural network model is a non-streaming network structure;
• and
• the outputting a first prediction result using the first multitask neural network model comprises:
• outputting the first prediction result based on a relationship between a total length of the to-be-recognized phoneme and a phoneme length threshold using the first multitask neural network model.

A non-transitory computer-readable storage medium having a computer program code stored thereon, wherein the computer program code comprises instructions that, when run on a processor of terminal device, cause the terminal device to perform steps including:

• inputting a to-be-recognized phoneme into a first multitask neural network model stored in one or more memories of the terminal device;
• automatically outputting a first prediction result based on execution by the processor of the first multitask neural network model applied to the to-be-recognized phoneme, wherein the first prediction result comprises a character prediction result and a punctuation prediction result each generated by the first multitask neural network model as applied to the to-be-recognized phoneme and corresponding to the to-be-recognized phoneme;
• and
• displaying at least a part of the first prediction result on a display of the terminal device based on the first prediction result;
• wherein:
• the first multitask neural network model is a streaming network structure;
• and
• the inputting, by a terminal device, a to-be-recognized phoneme into a first multitask neural network model, and outputting a first prediction result using the first multitask neural network model comprises:
• cyclically inputting, by the terminal device, the to-be-recognized phoneme into the first multitask neural network model, and outputting the first prediction result based on a length of a currently input to-be-recognized phoneme by using the first multitask neural network model;
• or
• the first multitask neural network model is a non-streaming network structure;
• and
• the outputting a first prediction result using the first multitask neural network model comprises:
• outputting the first prediction result based on a relationship between a total length of the to-be-recognized phoneme and a phoneme length threshold using the first multitask neural network model.