# US 10,484,801 B2 · Configuration of hearing prosthesis sound processor based on visual interaction with external device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,801 B2 - Document ID: US10484801B2 - Status: Granted - Kind: B2 - Application number: 14/851,900 - Filed: 2015-09-11 - Published: 2016-03-24 - Granted: 2019-11-19 - Earliest claimed: 2014-09-19 - Assignees: Cochlear Limited - Applicants: Cochlear Limited - Inventors: Yves Wernaers; Tom Van Assche - Web page: https://patentgrep.com/patent/US10484801B2 ## CPC - H04R 25/505: {using digital signal processing} - H04S 7/303: {Tracking of listener position or orientation} - H04R 2225/41: Detection or adaptation of hearing aid parameters or programs to listening situation, e.g. pub, forest - H04R 2225/43: Signal processing in hearing aids to enhance the speech intelligibility - H04R 2225/49: Reducing the effects of electromagnetic noise on the functioning of hearing aids, by, e.g. shielding, signal processing adaptation, selective (de)activation of electronic parts in hearing aid - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - H04S: STEREOPHONIC SYSTEMS ## Abstract As disclosed, an external device is associated with the recipient of a hearing prosthesis and provides to the hearing prosthesis a control signal carrying an indication that the recipient is looking at output of the external device, perhaps including an indication of one or more characteristics of visual output and/or audio output from the external device. In response to such a control signal, the hearing prosthesis then automatically configures its sound processor in a manner based at least in part on the provided indication, such as to help process audio coming from the external device and/or to process audio in a manner based on the indicated characteristic(s) of visual output and/or audio output. ## Claims ### Claim 1 A method comprising: receiving into a hearing prosthesis, from an external device associated with a recipient of the hearing prosthesis, a control signal indicating that the recipient is looking at an output of the external device; and responsive to receipt of the control signal, the hearing prosthesis automatically configuring a sound processor of the hearing prosthesis in a manner based at least in part on the control signal indicating that the recipient is looking at the output of the external device, wherein the external device is outputting visual content, the control signal indicates at least one characteristic of the visual content being output by the external device, and wherein automatically configuring the sound processor of the hearing prosthesis is further based at least in part on the control signal indicating the at least one characteristic of the visual content being output by the external device, and the at least one characteristic of the visual content comprises the visual content being text-based content, and wherein automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating the at least one characteristic of the visual content being output by the external device comprises at least one function that is at least one of: based at least in part on the control signal indicating that the visual content is text-based, automatically configuring the sound processor to reduce gain of the sound processor; based at least in part on the control signal indicating that the visual content is text-based, automatically reducing a stimulation rate of the sound processor; or based at least in part on the control signal indicating that the visual content is text-based, automatically increasing a noise reduction level of the sound processor. ### Claim 2 (depends on 1) The method of claim 1, wherein the control signal indicates that the recipient is looking at the output of the external device at least in part by indicating that the external device is outputting visual content, and wherein automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating that the recipient is looking at the output of the external device comprises: automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating that the external device is outputting visual content. ### Claim 3 (depends on 2) The method of claim 2, wherein the control signal indicates that the external device is outputting visual content at least in part by indicating that the external device is running an in-focus application of a type that outputs visual content. ### Claim 4 (depends on 1) The method of claim 1, wherein a subsequent signal received by the hearing prosthesis indicates at least one characteristic of a subsequent visual content being output by the external device, and the method further includes subsequently automatically configuring the sound processor of the hearing prosthesis based at least in part on the subsequent signal indicating the at least one characteristic of the subsequent visual content being output by the external device, and wherein the at least one characteristic of the visual content comprises image capture content, and wherein automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating the at least one characteristic of the visual content being output by the external device comprises: based at least in part on the control signal indicating that the visual content comprises image capture content, automatically configuring the sound processor to apply microphone-beamforming to focus on audio coming from in front of the recipient. ### Claim 5 (depends on 2) The method of claim 2, wherein the control signal further indicates that the external device is outputting audio content associated with the visual content, and wherein the automatic configuring of the sound processor is additionally based at least in part on the control signal indicating that the external device is outputting audio content associated with the visual content. ### Claim 6 (depends on 5) The method of claim 5, wherein automatically configuring the sound processor based at least in part on the control signal indicating that external device is outputting audio content associated with the visual content comprises: based at least in part on control signal indicating that external device is outputting audio content associated with the visual content, automatically configuring the sound processor to apply microphone-beamforming to focus on audio coming from the external device in front of the recipient. ### Claim 7 (depends on 5) The method of claim 5, wherein the control signal indicates at least one characteristic of the audio content being output by the external device, and wherein automatically configuring the sound processor of the hearing prosthesis is further based at least in part on the control signaling indicating the at least one characteristic of the audio content being output by the external device. ### Claim 8 (depends on 7) The method of claim 7, wherein the at least one characteristic of the audio content comprises the audio content being encoded with a particular codec, and wherein automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating the at least one characteristic of the audio content being output the by the external device comprises a function selected from the group consisting of: based at least in part on the control signal indicating that the audio content is encoded with the particular codec, automatically setting the sound processor to apply a particular bandpass filter, and based at least in part on the control signal indicating that the audio content is encoded with the particular codec, automatically modifying an extent of digital signal processing. ### Claim 9 (depends on 1) The method of claim 1, wherein the external device is a handheld computing device operable by the recipient. ### Claim 10 A hearing prosthesis comprising: at least one microphone for receiving audio input; a sound processor for processing the audio input and generating corresponding hearing stimulation signals to stimulate hearing in a human recipient of the hearing prosthesis; and a wireless communication interface, wherein the hearing prosthesis is configured to receive from an external device associated with the recipient, via the wireless communication interface, a control signal indicating that the recipient is focused at least visually on an output of the external device, and to respond to the received control signal by automatically configuring the sound processor in a manner based at least in part on the control signal indicating that the recipient is focused at least visually on the output of the external device, whereby the automatic configuring of the sound processor accommodates processing of the audio input when the audio input comprises audio content output from the external device. ### Claim 11 (depends on 10) The hearing prosthesis of claim 10, wherein the control signal indicates that the recipient is focused at least visually on the output of the external device at least in part by indicating that the external device is outputting visual content, and wherein automatically configuring the sound processor in a manner based at least in part on the control signal indicating that the recipient is focused at least visually on the output of the external device comprises automatically configuring the sound processor based at least in part on the control signal indicating that the external device is outputting visual content. ### Claim 12 (depends on 11) The hearing prosthesis of claim 11, wherein the control signal indicates that the external device is outputting visual content at least in part by indicating that the external device is running an in-focus application of a type that outputs visual content. ### Claim 13 (depends on 11) The hearing prosthesis of claim 11, wherein the control signal indicates at least one characteristic of the visual content being output by the external device, and wherein automatically configuring the sound processor is further based at least in part on the control signal indicating the at least one characteristic of the visual content being output by the external device. ### Claim 14 (depends on 10) The hearing prosthesis of claim 10, wherein the control signal further indicates that the external device is outputting the audio content, and wherein automatically configuring the sound processor is further based at least in part on the control signal indicating that the external device is outputting the audio content. ### Claim 15 A system comprising: a hearing prosthesis including a wireless communication interface, a microphone for receiving sound and a sound processor for processing received audio input, including audio input from the microphone, and generating hearing stimulation signals for a recipient of the hearing prosthesis; and a computing device associated with the recipient, wherein, the computing device is configured to output visual content viewable by the recipient, and wherein the computing device is configured to transmit to the hearing prosthesis a control signal indicating that the recipient is looking in a direction of the computing device, and wherein, the hearing prosthesis is configured to receive, via the wireless communication interface, the signal and in response, automatically configure the sound processor in a manner based at least in part on the control signal indicating that the recipient is looking in the direction of computing device so that a resulting hearing percept evoked by the hearing prosthesis based on sound originating from a location other than the computing device is different relative to that which would be the case in the absence of receipt of the signal. ### Claim 16 (depends on 15) The system of claim 15, wherein the control signal indicates that the recipient is looking in the direction of the computing device at least in part by indicating that the computing device is outputting visual content. ### Claim 17 (depends on 15) The system of claim 15, wherein the computing device is further configured to output audio content associated with the visual content, wherein the audio input represents at least the audio content, wherein the control signal further indicates at least one characteristic of the audio content, and wherein automatically configuring the sound processor is further based at least in part on the control signal indicating the at least one characteristic of the audio content. ### Claim 18 (depends on 1) The method of claim 1, wherein the control signal is not a configuration signal that configures the hearing prosthesis. ### Claim 19 (depends on 1) The method of claim 1, wherein the hearing prosthesis is pre-programmed to operate according to two or more respective operating regimes, at least one of which is automatically enabled upon the hearing prosthesis determining that it has received the control signal. ### Claim 20 (depends on 1) The method of claim 1, wherein the action of automatically configuring the sound processor is executed based on an analysis of the control signal by the hearing prosthesis in a non-slave manner vis-à-vis a master-slave control regime. ### Claim 21 (depends on 1) The method of claim 1, wherein the action of automatically configuring the sound processor is executed based on an analysis of the control signal by the hearing prosthesis in a slave manner vis-à-vis a master-slave control regime. ### Claim 22 (depends on 7) The method of claim 7, wherein the at least one characteristic of the audio content comprises the audio content including latency-sensitive audio content, and wherein automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating the at least one characteristic of the audio content being output by the external device comprises: based at least in part on the control signal indicating that the audio content comprises latency-sensitive audio content, automatically adjusting at least one sound processor setting to help reduce latency of sound processing. ### Claim 23 (depends on 22) The method of claim 22, wherein the latency-sensitive audio content comprises gaming audio content. ### Claim 24 (depends on 15) The system of claim 15, wherein the system is devoid of eye-tracking technology. ### Claim 25 (depends on 1) The method of claim 1, wherein: during the actions of receiving and automatically configuring, the sound processor of the hearing prosthesis is controlled by the hearing prosthesis. ### Claim 26 (depends on 1) The method of claim 1, wherein the action of receiving and automatically configuring are executed while the recipient is holding the external device while the recipient is looking at the device viewing text output. ### Claim 27 (depends on 1) The method of claim 1, further comprising: automatically configuring the sound processor of the hearing prosthesis in a different manner in the complete absence of the control signal, wherein the hearing prosthesis evokes a hearing percept while configured in the different manner. ### Claim 28 (depends on 1) The method of claim 1, wherein the control signal indicates that the recipient is looking at the output of the external device at least in part by indicating that the external device is outputting no visual content, and wherein automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating that the recipient is looking at the output of the external device comprises: automatically configuring the sound processor of the hearing prosthesis based at least in part on the control signal indicating that the external device is outputting visual content. ### Claim 29 (depends on 1) The method of claim 1, wherein the control signal further indicates that the external device is outputting audio content associated with the visual content, and wherein the automatic configuring of the sound processor is additionally based at least in part on the control signal indicating that the external device is outputting audio content associated with the visual content. ### Claim 30 (depends on 1) The method of claim 1, wherein the action of receiving and automatically configuring are executed while the recipient is holding the external device while the recipient is looking at the device watching video media content. ### Claim 31 (depends on 15) The system of claim 15, wherein the control signal indicates that the recipient is looking in the direction of the computing device at least in part by indicating that the computing device is outputting visual content, and wherein the computing device is further configured to output audio content associated with the outputted visual content, wherein the audio input represents at least the audio content, wherein the control signal further indicates at least one characteristic of the audio content, and wherein automatically configuring the sound processor is further based at least in part on the control signal indicating the at least one characteristic of the audio content. ### Claim 32 (depends on 1) The method of claim 1, wherein the external device is a television. ### Claim 33 (depends on 1) The method of claim 1, wherein the action of receiving and automatically configuring are executed while the hearing prosthesis operates autonomously from the external device.