Methods and systems for performing end-to-end spoken language analysis

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:

• receive, as input, an audio representation of speech having semantic characteristics;
• provide the audio representation to a spoken language understanding (SLU) network;
• extract, by the SLU network, filter-bank features of the audio representation;
• generate, by the SLU network, a saliency map based on the extracted filter-bank features;
• determine, by the SLU network, a first portion of the saliency map corresponds to a first high-importance feature and a second portion of the saliency map corresponds to a second high-importance feature different from the first high-importance feature, at least the first and the second high-importance features providing meaning to the semantic characteristics;
• infer, by the SLU network, intent of the speech directly from at least the first and the second high-importance features of the saliency map;
• and
• output, by the SLU network, a first response to the first high-importance feature and a second response different from the first response to the second high-importance feature.

An apparatus comprising:

• at least one hardware processor circuit operable to execute stored instructions that, when executed, causes the at least one hardware processor circuit to:
• receive an audio representation of speech having semantic characteristics;
• provide the audio representation to a spoken language understanding (SLU) network;
• extract, by the SLU network, filter-bank features of the audio representation;
• generate, by the SLU network, a saliency map based on the extracted filter-bank features;
• determine, by the SLU network, a first portion of the saliency map corresponds to a first high-importance feature and a second portion of the saliency map corresponds to a second high-importance feature different from the first high-importance feature, at least the first and the second high-importance features providing meaning to the semantic characteristics;
• infer, by the SLU network, intent of the speech directly from at least the first and the second high-importance features of the saliency map;
• and
• output, by the SLU network, a first response to the first high-importance feature and a second response different from the first response to the second high-importance feature.