# US 7,610,547 B2 · Markup language extensions for web enabled recognition > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,610,547 B2 - Document ID: US7610547B2 - Status: Granted - Kind: B2 - Application number: 10/117,141 - Filed: 2002-04-05 - Published: 2002-11-14 - Granted: 2009-10-27 - Earliest claimed: 2001-05-04 - Assignees: Microsoft Corporation - Inventors: Kuansan Wang; Hsiao-Wuen Hon - Web page: https://patentgrep.com/patent/US7610547B2 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 9/451: Execution arrangements for user interfaces - G06F 40/117: Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154) - G10L 15/24: Speech recognition using non-acoustical features - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - H04M 1/72445: for supporting Internet browser applications - H04M 3/493: Interactive information services, e.g. directory enquiries {; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals} - H04M 1/271: {controlled by voice recognition} - H04M 3/4936: {Speech interaction details (speech recognition per se G10L15/00)} - H04M 2207/40: terminals with audio html browser - H04M 2250/74: with voice recognition means - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract A markup language for execution on a client device in a client/server system includes extensions for recognition. ## Claims ### Claim 1 In combination with a remote recognition server, a client device including a computer readable storage medium having a markup language for execution on the client device in the client/server system, the markup language comprising an instruction to provide to the remote recognition server that is remote from the client device an indication of a grammar to associate with input entered through the client device and an instruction related to a mode of operation of the remote recognition server related to when recognition results are to be returned to the client device. ### Claim 2 (depends on 1) The computer readable storage medium of claim 1 wherein the instruction indicating a grammar provides a reference to a location of the grammar. ### Claim 3 (depends on 1) The computer readable storage medium of claim 1 wherein the markup language comprises one of HTML, XHTML, cHTML, XML, and WML. ### Claim 4 (depends on 1) The computer readable storage medium of claim 1 wherein the markup language comprises a scripting language. ### Claim 5 (depends on 1) The computer readable storage medium of claim 1 wherein the markup language comprises synchronized multimedia markup language. ### Claim 6 (depends on 1) The computer readable storage medium of claim 1 wherein the markup language includes an instruction to audibly render an indication when the speech was not recognized. ### Claim 7 (depends on 1) The computer readable storage medium of claim 1 wherein the markup language includes an instruction to visually render an indication when the speech was not recognized. ### Claim 8 (depends on 1) The computer readable storage medium of claim 1 wherein the instruction related to the mode of operation instructs the remote recognition server to provide recognition results when a time period related to speech recognition has lapsed. ### Claim 9 (depends on 1) The computer readable storage medium of claim 1 wherein the instruction related to the mode of operation instructs the remote recognition server to provide recognition results when a stop recognition indication is provided by the client device. ### Claim 10 (depends on 1) The computer readable storage medium of claim 1 wherein the instruction related to the mode of operation instructs the remote recognition server to start recognition when a start recognition indication is provided by the client device and to provide recognition results when a stop recognition indication is provided by the client device. ### Claim 11 (depends on 1) The computer readable storage medium of claim 1 wherein the instruction related to the mode of operation instructs the remote recognition server to start recognition when a start recognition indication is provided by the client device and to provide recognition results at intervals. ### Claim 12 (depends on 1) The computer readable storage medium of claim 1 wherein the markup language includes an instruction indicating a confidence measure to use for speech recognition. ### Claim 13 (depends on 1) The computer readable storage medium of claim 1 wherein the markup language includes an instruction for taking action based on a value previously received. ### Claim 14 A computer readable storage medium including instructions readable by a computer which, when implemented, cause the computer to handle information by performing a method comprising: receiving a markup language page from a web server having a field for input data; receiving input from a user related to the field; and sending to a remote recognition server markup language instructions for data indicative of the input, an indication of a grammar for recognition, a settable time period related to recognition to a remote recognition server, and an instruction related to a mode of operation of the remote recognition server related to when recognition results are to be returned. ### Claim 15 (depends on 14) The computer readable storage medium of claim 14 wherein the indication provides a reference to locations of the grammars. ### Claim 16 (depends on 14) The computer readable storage medium of claim 14 wherein the indication includes a reference to a language for speech recognition. ### Claim 17 (depends on 14) The computer readable storage medium of claim 14 wherein the markup language comprises one of HTML, XHTML, cHTML, XML and WML. ### Claim 18 (depends on 14) The computer readable storage medium of claim 14 wherein the markup language comprises a scripting language. ### Claim 19 (depends on 14) The computer readable storage medium of claim 14 wherein the markup language comprises a synchronized markup language. ### Claim 20 (depends on 14) The computer readable storage medium of claim 14 and wherein the input comprises speech, and further including instructions, which when implemented, comprise: processing the input speech to provide data indicative of the input speech. ### Claim 21 (depends on 20) The computer readable storage medium of claim 20 wherein processing includes normalizing the data indicative of the input speech. ### Claim 22 (depends on 20) The computer readable storage medium of claim 20 wherein processing includes generating data indicative of parameters of the input speech. ### Claim 23 (depends on 14) The computer readable storage medium of claim 14 wherein the input comprises speech and the grammar comprises a speech recognition grammar. ### Claim 24 (depends on 14) The computer readable storage medium of claim 14 wherein the input comprises DTMF input and the grammar comprises a DTMF grammar. ### Claim 25 (depends on 14) The computer readable storage medium of claim 14 wherein the instruction related to the mode of operation instructs the remote recognition server to provide recognition results when the time period related to speech recognition has lapsed. ### Claim 26 (depends on 14) The computer readable storage medium of claim 14 wherein the instruction related to the mode of operation instructs the remote recognition server to provide recognition results when a stop recognition indication is provided by the computer. ### Claim 27 (depends on 14) The computer readable storage medium of claim 14 wherein the instruction related to the mode of operation instructs the remote recognition server to start recognition when a start recognition indication is provided by the computer and to provide recognition results when a stop recognition indication is provided by the computer. ### Claim 28 (depends on 14) The computer readable storage medium of claim 14 wherein the instruction related to the mode of operation instructs the remote recognition server to start recognition when a start recognition indication is provided by the computer and to provide recognition results at intervals. ### Claim 29 In combination with a remote recognition server, a first client device and a second client device, each client device including a computer readable storage medium having a markup language for execution on the corresponding client device in the client/server system, the markup language comprising an instruction to provide to the remote recognition server that is remote from each of the client devices an indication of a grammar to associate with input entered through each client device, an instruction to set a time period related to speech recognition at the remote recognition server for the input provided from each client device, and an instruction related to a mode of operation of the remote recognition server related to when recognition results are to be returned to each of the client devices. ### Claim 30 (depends on 29) The computer readable storage medium of claim 29 wherein the instruction related to the mode of operation instructs the remote recognition server to provide recognition results when the time period related to speech recognition has lapsed. ### Claim 31 (depends on 30) The computer readable storage medium of claim 30 wherein the time period indicates a maximum period before a result must be returned after detection of speech. ### Claim 32 (depends on 30) The computer readable storage medium of claim 30 wherein the time period indicates a maximum period before a result must be returned after a start recognition instruction. ### Claim 33 (depends on 30) The computer readable storage medium of claim 30 wherein the time period indicates a minimum period of silence to indicate an end of speech. ### Claim 34 (depends on 30) The computer readable storage medium of claim 30 wherein the instruction related to the mode of operation instructs the remote recognition server to provide recognition results when a stop recognition indication is provided by the respective client device. ### Claim 35 (depends on 30) The computer readable storage medium of claim 30 wherein the instruction related to the mode of operation instructs the remote recognition server to start recognition when a start recognition indication is provided by the respective client device and to provide recognition results at intervals, wherein the time period related to speech recognition indicates a maximum time for providing recognition results. ### Claim 36 (depends on 29) The computer readable storage medium of claim 29 wherein the instruction related to the mode of operation instructs the remote recognition server to start recognition when a start recognition indication is provided by the respective client device and to provide recognition results when a stop recognition indication is provided by the respective client device. ### Claim 37 In combination with a remote recognition server, a first client device and a second client device, each client device including a computer readable storage medium having a markup language for execution on the corresponding client device in the client/server system, the markup language comprising an instruction to provide to the remote recognition server that is remote from each of the client devices an indication of a grammar to associate with input entered through each client device, and an instruction related to a mode of operation of the remote recognition server related to when recognition results are to be returned to each of the client devices, wherein the instruction related to the mode of operation of the remote recognition server for the first client device instructs the recognition server to provide recognition results only when a time period related to speech recognition has lapsed, and wherein the instruction related to the mode of operation of the remote recognition server for the second client device instructs the recognition server to provide recognition results when a stop recognition indication is provided by the second client device. ### Claim 38 (depends on 37) The computer readable storage medium of claim 37 wherein the instruction related to the mode of operation for the second client device instructs the remote recognition server to start recognition when a start recognition indication is provided by the second client device. ### Claim 39 (depends on 37) The computer readable storage medium of claim 37 wherein the instruction related to the mode of operation of the first client device instructs the remote recognition server to start recognition when a start recognition indication is provided by the respective client device and to provide recognition results at intervals, wherein the time period related to speech recognition indicates a maximum time for providing recognition results.