# US 8,788,620 B2 · Web service support for a multimodal client processing a multimodal application > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,788,620 B2 - Document ID: US8788620B2 - Status: Granted - Kind: B2 - Application number: 11/696,230 - Filed: 2007-04-04 - Published: 2008-10-09 - Granted: 2014-07-22 - Earliest claimed: 2007-04-04 - Assignees: International Business Machines Corporation - Applicants: Soonthorn Ativanichayaphong; Charles W. Cross, Jr.; Gerald M. McCobb - Inventors: Soonthorn Ativanichayaphong; Charles W. Cross, Jr.; Gerald M. McCobb - Web page: https://patentgrep.com/patent/US8788620B2 ## CPC - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - H04L 67/025: for remote control or remote monitoring of applications - H04L 69/329: in the application layer [OSI layer 7] - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Web service support for a multimodal client processing a multimodal application, the multimodal client providing an execution environment for the application and operating on a multimodal device supporting multiple modes of user interaction including a voice mode and one or more non-voice modes, the application stored on an application server, includes: receiving, by the server, an application request from the client that specifies the application and device characteristics; determining, by a multimodal adapter of the server, modality requirements for the application; selecting, by the adapter, a modality web service in dependence upon the modality requirements and the characteristics for the device; determining, by the adapter, whether the device supports VoIP in dependence upon the characteristics; providing, by the server, the application to the client; and providing, by the adapter to the client in dependence upon whether the device supports VoIP, access to the modality web service for processing the application. ## Claims ### Claim 1 A method of processing a voice command spoken into a communication device, the method comprising: establishing a connection via a communication network between an application server and the communication device; receiving, by the application server, the voice command spoken into the communication device and communicated to the application server via said connection; accessing a web service stored on the application server; processing the voice command with the web service on the application server; determining, by a multimodal adapter of the application server, modality requirements for a multimodal application operating on the communication device, the modality requirements specifying characteristics of a modality web service capable of processing the voice command; selecting, by the multimodal adapter of the application server, the modality web service in dependence upon the modality requirements; and returning, from the application server to the communication device, a multimodal signal configured to render a user interface on the communication device. ### Claim 2 (depends on 1) The method of claim 1, wherein the voice command is an application request to control the multimodal application operating on the communication device. ### Claim 3 (depends on 1) The method of claim 1, wherein the communication network is a data communication network and the connection is a VOIP connection. ### Claim 4 (depends on 1) The method of claim 1, wherein the receiving further comprises: receiving, by the application server, an application request from a multimodal client operating on the communication device, the application request specifying the multimodal application and characteristics for the communication device. ### Claim 5 (depends on 4) The method of claim 4, wherein the multimodal signal comprises a multimodal user interface configured to support the multimodal application operating on the communication device. ### Claim 6 (depends on 5) The method of claim 5, wherein the application request comprises user data for a user of the communication device, said user data including user preferences for customizing the user interface. ### Claim 7 (depends on 6) The method of claim 6, wherein the user data specifies interface components for the user interface including at least one of buttons, icons, lists, graphics, videos, voice dialogs, grammars and prompts. ### Claim 8 (depends on 1) The method of claim 1, wherein the web service stored on the application server is said modality web service; and wherein said characteristics include data communications capabilities of the communication device, codecs supported by the communication device, and language settings for the communication device. ### Claim 9 (depends on 1) The method of claim 1, further comprising: determining, by the multimodal adapter of the application server, whether the communication device supports Voice Over Internet Protocol (“VoIP”). ### Claim 10 (depends on 1) The method of claim 1, further comprising: encapsulating the modality web service into an HTTP message; wherein the returning of the multimodal signal comprises transmitting the HTTP message to the communication device in response to receiving the voice command. ### Claim 11 A system configure to process a voice command spoken into a communication device, the system comprising: a connection to a communication network, said connection being configure to communicate signals between an application server and the communication device; the application server being configured to receive the voice command spoken into the communication device and communicated to the application server via said connection; a memory of the application server configured to store a web service; a processor of the application server configured to process said voice command; a multimodal adapter stored on the application server and configured to determine modality requirements for a multimodal application operating on the communication device, the modality requirements specifying characteristics of a modality web service capable of processing the voice command and select the modality web service in dependence upon the modality requirements; and a communication port of the application server configured to return to the communication device a multimodal signal for rendering a user interface on the communication device. ### Claim 12 (depends on 11) The system of claim 11, wherein the voice command is an application request to control the multimodal application operating on the communication device. ### Claim 13 (depends on 11) The system of claim 11, wherein the communication network is a data communication network and the connection is a VOIP connection. ### Claim 14 (depends on 11) The system of claim 11, wherein the application server is further configured to receive an application request from a multimodal client operating on the communication device, the application request specifying the multimodal application and characteristics for the communication device. ### Claim 15 (depends on 14) The system of claim 14, wherein the web service stored on the application server is said modality web service; and wherein said characteristics include data communications capabilities of the communication device, codecs supported by the communication device, and language settings for the communication device. ### Claim 16 (depends on 11) The system of claim 11, wherein the multimodal adapter of the application server is further configured to determine whether the communication device supports Voice Over Internet Protocol (“VoIP”). ### Claim 17 (depends on 1) The system of claim 1, wherein the multimodal signal comprises a multimodal user interface configured to support the multimodal application operating on the communication device. ### Claim 18 (depends on 17) The system of claim 17, wherein the application request comprises user data for a user of the communication device, said user data including user preferences for customizing the user interface. ### Claim 19 (depends on 18) The system of claim 18, wherein the user data specifies interface components for the user interface including at least one of buttons, icons, lists, graphics, videos, voice dialogs, grammars and prompts. ### Claim 20 (depends on 11) The system of claim 11, wherein the application server encapsules the modality web service into an HTTP message; and wherein the returning of the multimodal signal comprises transmitting the HTTP message to the communication device in response to receiving the voice command.