# US 2019/0174002 A1 · Methods and Apparatus to Assist Listeners in Distinguishing Between Electronically Generated Binaural Sound and Physical Environment Sound > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0174002 A1 - Document ID: US20190174002A1 - Status: Published application - Kind: A1 - Application number: 16/271,784 - Filed: 2019-02-09 - Published: 2019-06-06 - Earliest claimed: 2016-06-10 - Inventors: Glen A. Norris; Philip Scott Lyren - Web page: https://patentgrep.com/patent/US20190174002A1 ## CPC - H04M 3/568: {audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants (echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - G10L 21/028: using properties of sound source - H04S 1/005: {For headphones} - H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)} - H04S 7/304: {For headphones} - H04S 7/305: {Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence)} - H04S 7/306: {For headphones} - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S 2400/13: Aspects of volume control, not necessarily automatic, in stereophonic sound systems - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04S: STEREOPHONIC SYSTEMS ## Abstract Methods and apparatus assist listeners in distinguishing between electronically generated binaural sound and physical environment sound while the listener wears a wearable electronic device that provides the binaural sound to the listener. The wearable electronic device generates a visual alert or audio alert when the electronically generated binaural sound occurs. ## Claims ### Claim at position 1 -20. (canceled) ### Claim 21 A method comprising: enabling a listener to distinguish between electronically generated binaural sound and physical environment sound during a telephone call with a person by: playing, with a wearable electronic device worn by the listener and during the telephone call, an audio alert that signifies that the electronically generated binaural sound is a voice of the person to assist the listener to distinguish between the voice of the person and the physical environment sound; and repeating, with the wearable electronic device worn by the listener, the audio alert during the telephone call to remind the listener that the electronically generated binaural sound is the voice of the person and not the physical environment sound. ### Claim 22 (depends on 21) The method of claim 21, further comprising: playing the audio alert at a point in time right before the listener hears the voice of the person so the listener knows the voice of the person is the electronically generated binaural sound and not the physical environment sound, wherein the wearable electronic device is one of a smartphone, electronic glasses, head mounted display, or headphones. ### Claim 23 (depends on 21) The method of claim 21, further comprising: playing the audio alert at a point in time when the electronically generated binaural sound finishes to signify completion of the electronically generated binaural sound, wherein the wearable electronic device is one of a smartphone, electronic glasses, head mounted display, or headphones. ### Claim 24 (depends on 21) The method of claim 21, further comprising: convolving, with a processor in the wearable electronic device and during the telephone call between the listener and the person, the audio alert with head-related transfer functions (HRTFs) so the audio alert externally localizes as binaural sound at least three feet away from a head of the listener; and repeating the audio alert during the telephone call. ### Claim 25 (depends on 21) The method of claim 21, further comprising: displaying, with the wearable electronic device, a visual alert that signifies to the listener that the electronically generated binaural sound is the voice of the person to assist the listener in distinguishing between the voice of the person and the physical environment sound. ### Claim 26 (depends on 21) The method of claim 21, further comprising: playing, with the wearable electronic device and during the telephone call, the audio alert as a tone in one of mono sound or stereo sound when the voice of the person externally localizes to the listener in binaural sound. ### Claim 27 (depends on 21) The method of claim 21, further comprising: providing, with the wearable electronic device and during the telephone call, the audio alert to the listener as a background sound when the person starts talking. ### Claim 28 A method comprising: displaying, with a display of a wearable electronic device worn by a listener during a telephone call between the listener and a user, an image that represents the user; playing, with the wearable electronic device worn by the listener during the telephone call, a voice of the user as binaural sound that externally localizes at least three feet away from a head of the listener; and enabling the listener to distinguish between the voice of the user that is the binaural sound during the telephone call and a voice emanating from a person in a physical environment where the listener is located by: displaying, with the display of the wearable electronic device worn by the listener during the telephone call, a visual alert that activates when the person in the physical environment speaks during the telephone call to enable the listener to distinguish between the voice of the user and the voice emanating from the person in the physical environment where the listener is located. ### Claim 29 (depends on 28) The method of claim 28, further comprising: deactivating, by the wearable electronic device worn by the listener during the telephone call, the visual alert when the person in the physical environment is not speaking to the listener during the telephone call such that the visual alert activates when the person is speaking and deactivates when the person is not speaking. ### Claim 30 (depends on 28) The method of claim 28, further comprising: displaying, with the wearable electronic device worn by the listener during the telephone call, the visual alert as one of a green light or a blue light with the display while the person is speaking to the listener during the telephone call to visual inform the listener when the person is speaking. ### Claim 31 (depends on 28) The method of claim 28, further comprising: displaying, with the display of the wearable electronic device worn by the listener during the telephone call, the visual alert as an image that represents the person. ### Claim 32 (depends on 28) The method of claim 28, further comprising: detecting, with microphones of the wearable electronic device worn by the listener during the telephone call, the voice emanating from the person in the physical environment where the listener is located; and displaying, with the display of the wearable electronic device worn by the listener during the telephone call and in response to detecting the voice emanating from the person, the visual alert while the person in the physical environment speaks. ### Claim 33 (depends on 28) The method of claim 28, further comprising: displaying, with the wearable electronic device worn by the listener during the telephone call, the visual alert as a three-dimensional (3D) image that is in a field-of-view of the listener; rotating, with the wearable electronic device worn by the listener during the telephone call, the 3D image while the person is speaking to the listener during the telephone call to visually indicate to the listener that the person is speaking; and stopping, with the wearable electronic device worn by the listener during the telephone call, rotation of the 3D image while the person is not speaking to the listener during the telephone call to visually indicate to the listener that the person is no longer speaking to the listener. ### Claim 34 (depends on 28) The method of claim 28, further comprising: detecting, with a sensor of the wearable electronic device worn by the listener during the telephone call, a gesture of a hand of the listener; changing, in response to detecting the gesture of the hand of the listener, the voice of the user from the binaural sound to one of stereo sound or mono sound; and providing, with the wearable electronic device worn by the listener during the telephone call, the voice of the user as the one of the stereo sound or the mono sound in response to detecting the gesture of the hand of the listener. ### Claim 35 A method executed by one or more electronic devices in a computer system to visually indicate to a first user when a second user talks to the first user during a telephone call between the first user and the second user, the method comprising: displaying, with a display of a wearable electronic device worn by the first user during the telephone call between the first user and the second user, an image that represents the second user in an augmented reality (AR) or virtual reality (VR) environment; processing, with a processor during the telephone call, a voice of the second user so the voice of the second user externally localizes as binaural sound to the first user at the image that represents the second user in the AR or VR environment; and visually indicating to the first user, with the display of the wearable electronic device worn by the first user, when the second user talks to the first user during the telephone call by activating a light in the AR or VR environment when the second user talks to the first user during the telephone call and deactivating the light in the AR or VR environment when the second user does not talking to the first user during the telephone call. ### Claim 36 (depends on 35) The method of claim 35, further comprising: visually indicating to the first user, with the display of the wearable electronic device worn by the first user, when the second user talks to the first user during the telephone call by animating an object that is the light in the AR or VR environment when the second user talks to the first user during the telephone call and stopping the animating of the object that is the light in the AR or VR environment when the second user does not talk to the first user during the telephone call. ### Claim 37 (depends on 35) The method of claim 35, wherein the light illuminates on the image that represents the second user in the AR or VR environment when the second user talks to the first user during the telephone call and turns off when the second user does not talk to the first user during the telephone call. ### Claim 38 (depends on 35) The method of claim 35, further comprising: detecting, with a microphone in the wearable electronic device worn by the first user and during the telephone call, a voice of a person not in the AR or VR environment but in a physical environment of the first user; and visually indicating to the first user, with the display of the wearable electronic device worn by the first user and during the telephone call, when the voice of the person not in the AR or VR environment but in the physical environment occurs by activating movement of a 3D image in the AR or VR environment in a field-of-view of the first user to provide a visual indication to the first user that the voice of the person does not emanate from the AR or VR environment but emanates from the physical environment of the first user. ### Claim 39 (depends on 35) The method of claim 35, further comprising: detecting, with a microphone in the wearable electronic device worn by the first user and during the telephone call, a voice of a person not in the AR or VR environment but in a physical environment of the first user; and visually indicating to the first user, with the display of the wearable electronic device worn by the first user and during the telephone call, that the voice of the person not in the AR or VR environment is talking by displaying an image that represents the person not the AR or VR environment along with the image that represents the second user in the AR or VR environment. ### Claim 40 (depends on 35) The method of claim 35, further comprising: detecting, with a microphone in the wearable electronic device worn by the first user and during the telephone call, a voice of a person not in the AR or VR environment but in a physical environment of the first user; and distinguishing to the first user, with the wearable electronic device worn by the first user and during the telephone call, that the voice of the person not in the AR or VR environment is talking by electronically modifying the voice of the person not in the AR or VR environment.