# US 12,112,096 B2 · Audio command interface for a multimedia device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,096 B2 - Document ID: US12112096B2 - Status: Granted - Kind: B2 - Application number: 17/130,807 - Filed: 2020-12-22 - Published: 2021-05-13 - Granted: 2024-10-08 - Earliest claimed: 2015-08-31 - Assignees: Roku, Inc. - Applicants: Roku, Inc. - Inventors: Frank Llewellyn Maker, III - Web page: https://patentgrep.com/patent/US12112096B2 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - H04N 21/42203: {sound input device, e.g. microphone} - H04N 21/42222: {Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/6581: {Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application} - H04N 21/64322: {IP} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract System, method, and computer product embodiments for processing audio commands are described. A multimedia device receives a digitized audio sequence from a control device. The digitized audio sequence is associated with a display command for controlling a display device. Then, the digitized audio sequence is converted into an audio signature that corresponds to the digitized audio sequence. The multimedia device sends the audio signature to a platform server configured to use the audio signature to process the display command. After, the multimedia device receives content information from the platform server resulting from processing the display command. Then, the multimedia device causes the display device to provide media content related to the content information. ## Claims ### Claim 1 A method, comprising: receiving, at a control device, an indication to activate an audio sensor; activating, responsive to receiving the indication, the audio sensor; detecting, by the audio sensor, a sound containing an audio command, wherein the audio command corresponds to a display command configured to control one of a multimedia device or a display device connected to the multimedia device, wherein the sound is detected from the multimedia device, and wherein the sound is associated with content currently displayed by the multimedia device; digitizing, by an audio processing component of the control device, the audio command to form a digitized audio sequence; and transmitting the digitized audio sequence to the multimedia device. ### Claim 2 (depends on 1) The method of claim 1, wherein the digitized audio sequence is configured to instruct the multimedia device to determine the display command to be transmitted to the display device. ### Claim 3 (depends on 1) The method of claim 1, wherein the audio command has a frequency below 20 kHz. ### Claim 4 (depends on 1) The method of claim 1, wherein the audio command has a frequency at or above 20 kHz. ### Claim 5 (depends on 1) The method of claim 1, wherein the indication is transmitted by an accelerometer of the control device. ### Claim 6 (depends on 1) The method of claim 1, wherein the indication is based on detecting a mobile device within proximity to the control device. ### Claim 7 (depends on 1) The method of claim 1, wherein the indication is a network packet received via a wake-on wireless network driver of the control device. ### Claim 8 (depends on 1) The method of claim 1, wherein the control device is integrated as a component of the multimedia device, and wherein the control device transmits the digitized audio sequence to another component of the multimedia device. ### Claim 9 (depends on 1) The method of claim 1, wherein detecting the sound containing the audio command further comprises: detecting a certain start sequence of frequencies at a starting point of the sound; and identifying the audio command in the sound based on the certain start sequence of frequencies. ### Claim 10 (depends on 1) The method of claim 1, further comprising: detecting an end point of the sound, wherein digitizing the audio command is based on detecting the end point of the sound. ### Claim 11 (depends on 10) The method of claim 10, wherein detecting the end point of the sound further comprises: detecting a certain end sequence of frequencies at the end point of the sound. ### Claim 12 (depends on 10) The method of claim 10, wherein detecting the end point of the sound further comprises: determining a predetermined period of time for the audio command. ### Claim 13 (depends on 12) The method of claim 12, wherein the predetermined period of time is specified by a certain start sequence of frequencies at a start of the sound. ### Claim 14 (depends on 1) The method of claim 1, wherein the display command is associated with requesting one of the multimedia device or the display device to provide additional content associated with the content currently displayed by the multimedia device. ### Claim 15 A control device, comprising: an audio sensor; a wireless network interface; and at least one processor processor configured to: receive an indication to activate the audio sensor; activate, responsive to receiving the indication, the audio sensor; detect, by the audio sensor, a sound containing an audio command, wherein the audio command corresponds to a display command configured to control one of a multimedia device or a display device connected to the multimedia device, wherein the sound is detected from the multimedia device, and wherein the sound is associated with content currently displayed by the multimedia device; digitize the audio command to form a digitized audio sequence; and transmit, via the wireless network interface, the digitized audio sequence to the multimedia device. ### Claim 16 (depends on 15) The control device of claim 15, wherein the digitized audio sequence is configured to instruct the multimedia device to determine the display command to be transmitted to the display device. ### Claim 17 (depends on 15) The control device of claim 15, wherein the indication is based on detecting a mobile device within proximity to the control device. ### Claim 18 (depends on 15) The control device of claim 15, wherein the indication is a network packet received via the wireless network interface. ### Claim 19 A non-transitory computer-readable medium having instructions stored thereon that, when executed by a control device, cause the at least one computing device to perform operations comprising: receiving, at the control device, an indication to activate an audio sensor; activating, responsive to receiving the indication, the audio sensor; detecting, by the audio sensor, a sound containing an audio command, wherein the audio command corresponds to a display command configured to control one of a multimedia device or a display device connected to the multimedia device, wherein the sound is detected from the multimedia device, and wherein the sound is associated with content currently displayed by the multimedia device; digitizing, by an audio processing component of the control device, the audio command to form a digitized audio sequence; and transmitting the digitized audio sequence to the multimedia device. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the digitized audio sequence is configured to instruct the multimedia device to determine the display command to be transmitted to the display device. ### Claim 21 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the indication is based on detecting a mobile device within proximity to the control device. ### Claim 22 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the indication is a network packet received via the wireless network interface. ### Claim 23 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein detecting the sound containing the audio command further comprises: detecting a certain start sequence of frequencies at a starting point of the sound; and identifying the audio command in the sound based on the certain start sequence of frequencies. ### Claim 24 (depends on 19) The non-transitory computer-readable medium of claim 19, the operations further comprising: detecting an end point of the sound, wherein digitizing the audio command is based on detecting the end point of the sound.