Speech recognition method and apparatus

An apparatus, comprising:

• at least one processor;
• and
• at least one computer-readable storage medium storing a program that is executable by the at least one processor, the program comprising instructions for:
• obtaining a first speech text;
• obtaining, based on the first speech text, first modal information that matches the first speech text, wherein a modality indicated by the first modal information is a first modality in a plurality of preset modalities;
• determining, based on the first speech text and the first modal information, a first intention and a first slot that are indicated by the first speech text when the first speech text matches the first modal information, wherein the first intention is an executable intention supported by the apparatus, and wherein the first slot is a slot holding data needed as input for executing the first intention;
• wherein the instructions for the obtaining, based on the first speech text, the first modal information that matches the first speech text comprise instructions for:
• obtaining a multimodal selection vector based on the first speech text;
• and
• obtaining the first modal information based on the multimodal selection vector;
• and
• wherein the instructions for the obtaining the multimodal selection vector based on the first speech text comprises:
• determining a first context category to which the first speech text belongs;
• and
• obtaining the multimodal selection vector based on the first context category and a first mapping matrix, wherein the multimodal selection vector indicates a probability of relevance between the first context category and each of the plurality of preset modalities, wherein the first mapping matrix indicates a plurality of context categories and a plurality of multimodal selection vectors, wherein each multimodal selection vector indicates one or more modalities, and wherein categories of the plurality of context categories are in a one-to-one correspondence with multimodal selection vectors of the plurality of multimodal selection vectors.

A non-transitory computer readable storage medium storing instructions that are executable by at least one processor, the instructions comprising instructions for:

• obtaining a first speech text;
• obtaining, based on the first speech text, first modal information that matches the first speech text, wherein a modality indicated by the first modal information is a first modality in a plurality of preset modalities;
• and
• determining, based on the first speech text and the first modal information, a first intention and a first slot that are indicated by the first speech text when the first speech text matches the first modal information, wherein the first intention is an executable intention supported by a system performing the instructions, and wherein the first slot is a slot holding data needed as input for executing the first intention;
• wherein the obtaining, based on the first speech text, the first modal information that matches the first speech text comprises:
• obtaining a multimodal selection vector based on the first speech text;
• and
• obtaining the first modal information based on the multimodal selection vector;
• and
• wherein the obtaining the multimodal selection vector based on the first speech text comprises:
• determining a first context category to which the first speech text belongs;
• and
• obtaining the multimodal selection vector based on the first context category and a first mapping matrix, wherein the multimodal selection vector indicates a probability of relevance between the first context category and each of the plurality of preset modalities, wherein the first mapping matrix indicates a plurality of context categories and a plurality of multimodal selection vectors, wherein each multimodal selection vector indicates one or more modalities, and wherein categories of the plurality of context categories are in a one-to-one correspondence with multimodal selection vectors of the plurality of multimodal selection vectors.