Rapid automatic user training with simulated bilingual user actions and responses in speech-to-speech translation

A non-transitory computer readable storage medium comprising a computer readable program, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:

• integrating an automatic user response system for training a user to operate a speech-to-speech translation system, wherein said automatic user response system is configured to be responsive to a plurality of training items;
• selecting a training item from the plurality of training items;
• for the selected training item:
• in response to an utterance in a first language, translating the utterance into a second language;
• generating a response to the utterance in the second language;
• and
• generating a simulated action corresponding with the response in accordance with a user speaking the second language where the response and simulated action are output as a learning exercise for learning operations of the speech-to-speech translation system.

A method for automatic user training in speech-to-speech translation, comprising:

• collecting dialogues and actions for speech translation applications;
• training models and establishing rules for automatic generation of multilingual responses;
• training models and establishing rules for automatic generation of user actions;
• integrating an automatic user response system for training a user to operate a speech-to-speech translation system, wherein said automatic user response system is configured to be responsive to a plurality of training items;
• selecting a training item from the plurality of training items such that for the selected training item:
• in response to an utterance in a first language, translating the utterance into a second language;
• generating a response to the utterance in the second language;
• generating a simulated action corresponding with the response in accordance with a model for a user speaking the second language;
• and
• outputting the response and simulated action as a learning exercise for learning operations of the speech-to-speech translation system.

A translation system, comprising:

• a memory configured to store one or more training models and rules, the training models and rules configured for automatic generation of multilingual responses and for automatic generation of user actions;
• an automatic speech recognition engine and machine translation module configured to translate an utterance in a first language into a second language;
• an automatic user response system for training a user to operate a speech-to-speech translation system, wherein said automatic user response system is configured to employ the training models and rules to generate a response to the utterance in the second language and generate a simulated action corresponding with the response in accordance with a user speaking the second language;
• and
• an output device configured to output the response and simulated action as a learning exercise for learning operations of the speech-to-speech translation system.