# US 2017/0289712 A1 · A method for operating a hearing system as well as a hearing system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0289712 A1 - Document ID: US20170289712A1 - Status: Published application - Kind: A1 - Application number: 15/510,310 - Filed: 2014-09-12 - Published: 2017-10-05 - Earliest claimed: 2014-09-12 - Assignees: Sonova AG - Applicants: Sonova AG - Inventors: Manuela Feilner; Michael Boretzki; Harald Krueger - Web page: https://patentgrep.com/patent/US20170289712A1 ## CPC - H04R 25/558: {Remote control, e.g. of amplification, frequency} - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/172: {Classification, e.g. identification} - G10L 15/24: Speech recognition using non-acoustical features - H04R 25/407: {Circuits for combining signals of a plurality of transducers} - H04R 25/552: {Binaural} - H04R 25/405: {by combining a plurality of transducers} - H04R 2225/43: Signal processing in hearing aids to enhance the speech intelligibility - H04R 2225/55: Communication between hearing aids and external devices via a network for data exchange - H04R 2430/20: Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic (H04R2203/12 takes precedence) - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A method for operating a hearing system including a hearing device, a camera and an auxiliary device, the method including the steps of providing an input signal to the hearing device, capturing an image or a sequence of images of at least sections of a surrounding of a user wearing the hearing device, processing the image or the sequence of images in the auxiliary device) for obtaining consolidated data of a sound source being important for the user, transmitting the consolidated data to the hearing device, generating an output signal in the hearing device by processing the audio signal and by taking into account the consolidated data, and feeding the output signal to an output transducer of the hearing device. Thereby, a hearing system having improved capabilities is obtained. ## Claims ### Claim 1 A method for operating a hearing system comprising a hearing device (1), a camera (11,12) and an auxiliary device (21),the method comprising the steps of: providing an input signal for said hearing device (1), capturing an image or a sequence of images of at least sections of a surrounding of a user (10) wearing said hearing device (1), processing said image or said sequence of images in said auxiliary device (21) for obtaining consolidated data of a sound source (15, 22) being important for said user (10), transmitting said consolidated data to the hearing device (1), generating an output signal in said hearing device (1) by processing the input signal and by taking into account said consolidated data, and feeding said output signal to an output transducer of said hearing device (1). ### Claim 2 (depends on 1) The method of claim 1, wherein said processing comprises locating said sound source (15, 22) being of importance to said user (10) and said consolidated data of a sound source comprises a directional angle (α) being defined between a sagittal plane of said user (10) and said sound source (15, 22) seen from said user (10). ### Claim 3 (depends on 1) The method of claim 1, further comprising the steps of: selecting said sound source (15,22) being important to said user (10) of said hearing device (1) out of a plurality of sound sources (15, . . . , 19; 22; 24), and tagging said sound source (15; 24) by activating an input on the auxiliary device (21). ### Claim 4 (depends on 1) The method of claim 1, further comprising the step of tracking said sound source (15; 24). ### Claim 5 (depends on 1) The method of claim 1, wherein the auxiliary device (21) is one of the following: a smartphone; a remote control. ### Claim 6 (depends on 1) The method of claim 1, wherein said camera (11, 12) is positioned at the head of the user (10). ### Claim 7 (depends on 1) The method of claim 1, further comprising the step of augmenting said consolidated data by information of a sensor unit (14), such as a compass or an additional microphone signal. ### Claim 8 (depends on 1) The method of claim 1, further comprising the steps of: detecting a present position of said user (10) relative to the camera (11, 12), and calculating a direction (α) in which said sound source (15; 24) being important for said user (10) taking into account the present position of said user (10). ### Claim 9 (depends on 1) The method of claim 1, further comprising the steps of: detecting a favorite speaker as said sound source (10) by face recognition and comparison to a corresponding data base, and tracking said favorite speaker after a tracking command is received by said auxiliary device (21). ### Claim 10 (depends on 1) The method of claim 1, wherein said input signal to the hearing device (1) originate from at least one input transducer (5,6) or from a streaming unit (23). ### Claim 11 A hearing system comprising: a hearing device (1) comprising an input transducer (5,6), an output transducer (7) and a processing unit (3)being operatively connected to the input transducer (5, 6) and the output transducer (7), at least one camera (11, 12) for capturing an image or a sequence of images of at least a section of a surrounding of a user (10)wearing said hearing device (1). an auxiliary device (21) receiving and processing said image or said sequence of images for obtaining consolidated data of a sound source (15; 24) being important for said user (10), and a transmission channel between said auxiliary device (21) and said hearing device (10) for transmitting said consolidated data to said hearing device (1), wherein said processing unit (3) is adapted to take into account said consolidated data while generating an output signal for the output transducer (7). ### Claim 12 (depends on 11) The hearing system of claim 11, wherein a low power technology is used for the transmission channel, such as Bluetooth standard or inductive coupling. ### Claim 13 (depends on 11) The hearing system of claim 11, wherein the transmission channel is adapted to continuously transmit consolidated data from the auxiliary device (21) to the hearing device (1). ### Claim 14 (depends on 11) The hearing system of claim 11, wherein the auxiliary device (21) is a smartphone or a remote control.