Slot filling in spoken language understanding with joint pointer and attention

A system for spoken language understanding comprising:

• an input device;
• an output device;
• a memory;
• and
• a processor operatively connected to the input device, the output device, and the memory, the processor being configured to:
• receive an input sequence including a plurality of words in a text phrase based on an input received with the input device;
• generate an encoded output corresponding to the plurality of words and a plurality of attention weights corresponding to the plurality of words using a recurrent neural network (RNN) encoder with an attention mechanism stored in the memory that receives the plurality of words as inputs to the RNN encoder;
• generate a first probability distribution corresponding to a plurality of words in an extended slot vocabulary database stored in the memory being a slot word at a time step using an RNN decoder stored in the memory that receives the encoded output from the RNN encoder as input;
• generate a second probability distribution corresponding to the plurality of words in the input sequence being a slot word at the time step using a pointer network that receives the plurality of attention weights as input;
• generate a combined probability distribution based on the first probability distribution and the second probability distribution;
• identify a slot word corresponding to one word in the extended slot vocabulary database having a highest probability value in the combined probability distribution for the time step;
• and
• perform a command using the slot word as a parameter to the command to produce an output with the output device.