System and method for generating and presenting multi-modal applications from intent-based markup scripts

A method for providing global help information when presenting a modality-independent document, the method comprising the steps of:

• preparing an internal representation of a structure and component attributes of the modality-independent document;
• building a grammar comprising rules for resolving specific spoken requests;
• processing a spoken request utilizing the grammar rules;
• and
• presenting an aural description of the modality-independent document in response to the spoken request.

A method for providing contextual help information when presenting a modality-independent document, the method comprising the steps of:

• preparing an internal representation of a structure and component attributes of the modality-independent document;
• building a grammar comprising rules for resolving specific spoken requests;
• processing a spoken request utilizing the grammar rules;
• and
• presenting an aural description of the components, attributes, and methods of interaction of the modality-independent document in response to the spoken request.

A method for providing feedback information when presenting a modality-independent document, the method comprising the steps of:

• preparing an internal representation of the structure and component attributes of the modality-independent document;
• building a grammar comprising rules for resolving specific spoken requests;
• processing a spoken request and resolving the spoken request utilizing the grammar rules;
• obtaining state and value information regarding specified components of the document from the internal representation of the document;
• and
• presenting an aural description of the content values associated with document components in response to the spoken request.

A method for aurally spelling out content values associated with components of a modality-independent document, the method comprising the steps of:

• preparing an internal representation of a structure and component attributes of the modality-independent document;
• building a grammar comprising rules for resolving specific spoken requests;
• processing a spoken request utilizing the grammar rules;
• obtaining state and content value information regarding specified components of the document from the internal representation of the document;
• and
• presenting each character of the content value information requested in response to the spoken request.

A system for presenting an application in a plurality of modalities, comprising:

• a multi-modal manager for parsing a modality-independent document to generate a traversal model that maps components of the modality-independent document to at least a first and second modality-specific representation;
• a speech user interface manager for rendering and presenting the first modality-specific representation in a speech modality;
• a GUI (graphical user interface) manager for rendering and presenting the second modality-specific representation in a GUI modality;
• an event queue monitor for detecting GUI events;
• an event queue for storing captured GUI events;
• and
• a plurality of methods, that are called by the speech user interface manager, for synchronizing I/O (input/output) events across the speech and GUI modalities.