Synchronizing visual and speech events in a multimodal application

A system, comprising:

• at least one computer processor;
• at least one computer memory operatively coupled to the computer processor;
• and
• computer program instructions disposed within the computer memory that, when executed, cause the at least one computer processor to:
• receive, by a multimodal application executing on a computer processor, multimodal input from a multimodal browser of a device, wherein the multimodal input comprises speech from a user;
• determine a semantic interpretation of at least a portion of the speech using a voice form, the semantic interpretation comprising a plurality of semantic interpretations of the at least a portion of the speech;
• call a global application update handler of the multimodal application;
• identify, by the global application update handler, an additional processing function based at least in part upon the semantic interpretation and a geographical location, for each of the plurality of semantic interpretations, wherein the additional processing function is independent of the voice form;
• and
• execute the additional processing function, wherein the additional processing function executed depends on the semantic interpretation of the at least a portion of the speech.

A non-transitory computer-readable storage medium comprising instructions that, when executed on at least one computer processor, perform a method, comprising:

• receiving, by a multimodal application executing on a computer processor, multimodal input from a multimodal browser of a device, wherein the multimodal input comprises speech from a user;
• determining a semantic interpretation of at least a portion of the speech using a voice form;
• calling a global application update handler of the multimodal application;
• identifying, by the global application update handler, an additional processing function based at least in part upon the semantic interpretation and a geographical location, wherein the additional processing function is independent of the voice form;
• and
• executing the additional processing function, wherein the additional processing function executed depends on the semantic interpretation of the at least a portion of the speech,
• wherein determining a semantic interpretation of at least a portion of the speech comprises determining a plurality of semantic interpretations of the at least a portion of the speech, and
• wherein identifying, by the global application update handler, an additional processing function in dependence upon the semantic interpretation comprises identifying, by the global application update handler, an additional processing function for each of the plurality of semantic interpretations.

A non-transitory computer-readable storage medium comprising instructions that, when executed on at least one computer processor, perform a method, comprising:

• receiving speech from a user;
• determining a semantic interpretation of at least a portion of the speech;
• and
• identifying, by a global application update handler, an additional processing function in dependence upon the semantic interpretation;
• executing the additional processing function to provide an advertisement based at least in part upon the semantic interpretation and a geographical location,
• wherein determining a semantic interpretation of at least a portion of the speech comprises determining a plurality of semantic interpretations of the at least a portion of the speech, and
• wherein identifying, by the global application update handler, an additional processing function in dependence upon the semantic interpretation comprises identifying, by the global application update handler, an additional processing function for each of the plurality of semantic interpretations.