# US 2017/0255446 A1 · Voice Control Of Interactive Whiteboard Appliances > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0255446 A1 - Document ID: US20170255446A1 - Status: Published application - Kind: A1 - Application number: 15/061,806 - Filed: 2016-03-04 - Published: 2017-09-07 - Earliest claimed: 2016-03-04 - Assignees: Ricoh Company, Ltd. - Inventors: Rathnakara Malatesha; Lana Wong; Hiroshi Kitada - Web page: https://patentgrep.com/patent/US20170255446A1 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 3/03545: {Pens or stylus} - G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer - G10L 2015/223: {Execution procedure of a spoken command} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A method and apparatus for providing voice command functionality to an interactive whiteboard appliance is provided. An interactive whiteboard appliance comprises: one or more processors; a non-transitory computer-readable medium having instructions embodied thereon, the instructions when executed by the one or more processors cause performance of: detecting, during execution of an annotation window on the interactive whiteboard appliance, a voice input received from a user; storing, in an audio packet, a recording of the voice input; transmitting the audio packet to a speech-to-text service; receiving, from the speech-to-text service, a command string comprising a transcription of the recording of the voice input; using voice mode command processing in a command processor, identifying, from the command string, an executable command that is executable by the interactive whiteboard appliance; causing the application of the interactive whiteboard appliance to execute the executable command. ## Claims ### Claim 1 An interactive whiteboard appliance comprising: one or more processors; a non-transitory computer-readable medium having instructions embodied thereon, the instructions when executed by the one or more processors cause performance of: detecting, during execution of an annotation window on the interactive whiteboard appliance, a voice input received from a user; storing, in an audio packet, a recording of the voice input; transmitting the audio packet to a speech-to-text service; receiving, from the speech-to-text service, a command string comprising a transcription of the recording of the voice input; using voice mode command processing in a command processor, identifying, from the command string, an executable command that is executable by the interactive whiteboard appliance; causing the application of the interactive whiteboard appliance to execute the executable command. ### Claim 2 (depends on 1) The interactive whiteboard appliance of claim 1, wherein the instructions, when executed, further cause performance of: detecting, during execution of the annotation window on an interactive whiteboard appliance, a stylus input from a user in the annotation window; using pen mode command processing in the command processor, identifying, from the stylus input, an additional command; causing the application of the interactive whiteboard appliance to execute the additional command. ### Claim 3 (depends on 1) The interactive whiteboard appliance of claim 1, wherein detecting the voice input comprises: operating a microphone in a continuous listening mode that captures a continuous audio feed; capturing the voice input from the continuous audio feed. ### Claim 4 (depends on 1) The interactive whiteboard appliance of claim 1, wherein detecting the voice input comprises: operating a microphone in a start command mode; detecting a start command; in response to detecting the start command, capturing the voice input from a subsequent audio feed. ### Claim 5 (depends on 1) The interactive whiteboard appliance of claim 1, wherein receiving, from the speech-to-text service, the command string comprises: using a client device associated with the interactive whiteboard appliance, detecting, during execution of an application on the client device associated with the interactive whiteboard appliance, a second voice input from a second user; using the client device, storing, in a second audio packet, a recording of the second voice input; using the client device, transmitting the second audio packet to a speech-to-text service; wherein the instructions, when executed, further cause performance of: at the interactive whiteboard appliance, receiving, from the speech-to-text service, a command string comprising a transcription of the recording of the second voice input; at the interactive whiteboard appliance, using voice mode command processing in the command processor, identifying, from the transcription of the recording of the second voice input, a second executable command; at the interactive whiteboard appliance, causing the application of the interactive whiteboard appliance to execute the second executable command. ### Claim 6 (depends on 5) The interactive whiteboard appliance of claim 5, wherein the client device is associated with the interactive whiteboard appliance using a selection of the interactive whiteboard appliance received from the user via the application on the mobile device. ### Claim 7 (depends on 5) The interactive whiteboard appliance of claim 5, wherein the client device comprises a mobile device. ### Claim 8 (depends on 5) The interactive whiteboard appliance of claim 5, wherein the client device comprises another interactive whiteboard appliance. ### Claim 9 (depends on 1) The interactive whiteboard appliance of claim 1, wherein the audio packet includes an identifier of the interactive whiteboard appliance that identifies the interactive whiteboard appliance from a plurality of interactive whiteboard appliances. ### Claim 10 (depends on 9) The interactive whiteboard appliance of claim 9, wherein transmitting the audio packet to the speech-to-text service comprises: transmitting the audio packet via an audio packet queue that stores a plurality of queued audio packets transmitted by the plurality of interactive whiteboard appliances. ### Claim 11 (depends on 9) The interactive whiteboard appliance of claim 9, wherein receiving, from the speech-to-text service, the command string comprises: receiving the command string via a command string queue stores a plurality of queued command strings to be sent to the plurality of interactive whiteboard appliances; determining that the identifier of the interactive whiteboard appliance included in the audio packet matches the identifier of the interactive whiteboard appliance. ### Claim 12 (depends on 1) The interactive whiteboard appliance of claim 1, wherein identifying, from the transcription of the recording of the voice input, the executable command comprises: modifying the transcription of the recording by adding or removing one or more characters from the transcription resulting in a modified transcription; matching the modified transcription to an executable command. ### Claim 13 (depends on 1) The interactive whiteboard appliance of claim 1, wherein causing the application of the interactive whiteboard appliance to execute the executable command comprises: communicating the executable command to an operating system of the interactive whiteboard appliance via an application programming interface (API). ### Claim 14 (depends on 1) The interactive whiteboard appliance of claim 1, wherein the speech-to-text service is implemented on the interactive whiteboard appliance. ### Claim 15 A method comprising: detecting, during execution of an annotation window on a interactive whiteboard appliance, a voice input received from a user; storing, in an audio packet, a recording of the voice input; transmitting the audio packet to a speech-to-text service; receiving, from the speech-to-text service, a command string comprising a transcription of the recording of the voice input; using voice mode command processing in a command processor, identifying, from the command string, an executable command that is executable by the interactive whiteboard appliance; causing the application of the interactive whiteboard appliance to execute the executable command. ### Claim 16 (depends on 15) The method of claim 15, wherein receiving, from the speech-to-text service, the command string comprises: using a client device associated with the interactive whiteboard appliance, detecting, during execution of an application on the client device associated with the interactive whiteboard appliance, a second voice input from a second user; using the client device, storing, in a second audio packet, a recording of the second voice input; using the client device, transmitting the second audio packet to a speech-to-text service; wherein the instructions, when executed, further cause performance of: at the interactive whiteboard appliance, receiving, from the speech-to-text service, a command string comprising a transcription of the recording of the second voice input; at the interactive whiteboard appliance, using voice mode command processing in the command processor, identifying, from the transcription of the recording of the second voice input, a second executable command; at the interactive whiteboard appliance, causing the application of the interactive whiteboard appliance to execute the second executable command. ### Claim 17 (depends on 16) The method of claim 16, wherein the client device is associated with the interactive whiteboard appliance using a selection of the interactive whiteboard appliance received from the user via the application on the mobile device. ### Claim 18 (depends on 16) The method of claim 16, wherein the client device comprises a mobile device. ### Claim 19 (depends on 16) The method of claim 16, wherein the client device comprises another interactive whiteboard appliance. ### Claim 20 A non-transitory computer-readable medium having instructions embodied thereon, the instructions when executed by the one or more processors cause performance of: detecting, during execution of an annotation window on an interactive whiteboard appliance, a voice input received from a user; storing, in an audio packet, a recording of the voice input; transmitting the audio packet to a speech-to-text service; receiving, from the speech-to-text service, a command string comprising a transcription of the recording of the voice input; using voice mode command processing in a command processor, identifying, from the command string, an executable command that is executable by the interactive whiteboard appliance; causing the application of the interactive whiteboard appliance to execute the executable command.