Dialog understanding device and dialog understanding method

A dialog understanding method for use in a dialog understanding device including:

• a sound collection module,
• a contextual processing module, and
• a dialog system, wherein the contextual processing module comprises three layers for processing information obtained from the sound collection module,
• wherein a fallback processing module, of a case where a predetermined process has not succeeded for the collected sound signal, is provided on each layer of the three layers,
• wherein the sound collection module collects a sound signal,
• wherein the dialog system performs a dialog with a human,
• wherein the contextual processing module performs a process of the next layer after a fallback corresponding to a fallback process performed by the fallback processing module is performed,
• wherein the contextual processing module inputs a sound signal obtained when the contextual processing module completes a process to the dialog system, wherein a first layer of the three layers comprises:
• an audio trigger module that detects a wake-up word, and
• a sound module that performs a sound source localization process and a sound source separation process,
• wherein a second layer of the three layers comprises:
• an automatic speech recognition module that converts speech into text with respect to a sound signal of a processing result of the first layer,
• a speaker recognition module that performs a speaker diarization process for identifying a speaker from an audio stream,
• a model adaptation module that performs speech-to-text model adaptation corresponding to a context-specific input, and
• a language detection module that detects a type of language recognized with reference to a language model provided in the language detection module, and
• wherein a third layer of the three layers comprises:
• a sentence processing module that classifies a type of sentence obtained in a textualization process in the second layer;
• and
• a sentence determination module that detects a type of entity from the sentence obtained in the textualization process.