# US 2013/0154930 A1 · Gesture controlled audio user interface > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2013/0154930 A1 - Document ID: US20130154930A1 - Status: Published application - Kind: A1 - Application number: 13/664,281 - Filed: 2012-10-30 - Published: 2013-06-20 - Earliest claimed: 2011-12-19 - Assignees: QUALCOMM Incorporated - Inventors: Pei Xiang; Hui-ya Liao Nelson - Web page: https://patentgrep.com/patent/US20130154930A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 1/1684: {Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 3/03547: {Touch pads, in which fingers can move on a surface} - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A user interface, methods and article of manufacture each for selecting an audio cue presented in three-dimensional (3D) space are disclosed. The audio cues are audibly perceivable in a space about a user, where each of the audio cues may be perceived by the user as a directional sound at a distinct location from other audio cues in the space. Selection of a specific audio cue is made based on one or more user gestures. A portable electronic device may be configured to present the audio cues perceived by a user and detect certain user gestures to select audio cues. The audio cue selection can be used to control operation of the portable device and/or other associated devices. ## Claims ### Claim 1 An apparatus, comprising: a circuit configured to generate a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space; a user interface configured to detect user movement relative to the apparatus, made in response to the audio cues; and a selector configured to select at least one of the audio cues based on the user movement. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the selector matches a direction of the user movement to one or more of the audio cues. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the circuit is further configured to produce an audio feedback cue based on the user movement detected by the user interface, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the apparatus further comprises a touch screen, and the user movement is a directional swipe across the touch screen of the apparatus. ### Claim 5 (depends on 1) The apparatus of claim 1, further comprising one or more sensors configured to detect an orientation of the apparatus. ### Claim 6 (depends on 5) The apparatus of claim 5, further comprising a processor configured to determine an absolute direction of a user directional swipe, irrespective of the orientation of the apparatus, based on output from the sensors. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the user interface includes one or more ultrasonic transducers configured to detect the user movement. ### Claim 8 (depends on 7) The apparatus of claim 7, wherein in the ultrasonic transducers are located along the periphery of the apparatus. ### Claim 9 (depends on 1) The apparatus of claim 1, further comprising: a speaker array configured to emit an ultrasonic signal; a portable device including a plurality of microphones producing microphone signals in response to the ultrasonic signal; and a processor configured to detect the user movement based on the microphone signals. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein the processor is also configured to determine the orientation of the portable device based on the microphone signals. ### Claim 11 (depends on 1) The apparatus of claim 1, further comprising: a portable device including a speaker array configured to emit an ultrasonic signal; a plurality of microphones producing microphone signals in response to the ultrasonic signal; and a processor configured to detect the user movement based on the microphone signals. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the processor is also configured to determine the orientation of the portable device based on the microphone signals. ### Claim 13 An apparatus, comprising: means for generating a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space; means for detecting user movement relative to the apparatus, made in response to the audio cues; and means for selecting at least one of the audio cues based on the user movement. ### Claim 14 (depends on 13) The apparatus of claim 13, further comprising means for matching a direction of the user movement to one or more of the audio cues. ### Claim 15 (depends on 13) The apparatus of claim 13, further comprising means for producing an audio feedback cue based on the user movement detected by the detecting means, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement. ### Claim 16 (depends on 13) The apparatus of claim 13, wherein the apparatus further comprising means for detecting a directional swipe across a touch screen included in the apparatus. ### Claim 17 (depends on 13) The apparatus of claim 13, further comprising means for detecting an orientation of the apparatus. ### Claim 18 (depends on 17) The apparatus of claim 17, further comprising means for determining an absolute direction of a user directional swipe, irrespective of the orientation of the apparatus. ### Claim 19 (depends on 13) The apparatus of claim 13, further comprising means for detecting the user movement with ultrasonic sensors. ### Claim 20 (depends on 13) The apparatus of claim 13, further comprising: means for emitting an ultrasonic signal; means for producing microphone signals in response to the ultrasonic signal; and means for detecting the user movement based on the microphone signals. ### Claim 21 (depends on 20) The apparatus of claim 20, further comprising means for determining the orientation of a portable device based on the microphone signals. ### Claim 22 A non-transitory computer-readable medium embodying a set of instructions executable by one or more processors, comprising: code for generating a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space; code for detecting user movement relative to a portable device, made in response to the audio cues; and code for selecting at least one of the audio cues based on the user movement. ### Claim 23 (depends on 22) The computer-readable medium of claim 22, further comprising code for matching a direction of the user movement to one or more of the audio cues. ### Claim 24 (depends on 22) The computer-readable medium of claim 22, further comprising code for producing an audio feedback cue based on detected user movement, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement. ### Claim 25 (depends on 22) The computer-readable medium of claim 22, further comprising code for determining a directional swipe across a touch screen included in the portable device. ### Claim 26 (depends on 22) The computer-readable medium of claim 22, further comprising code for detecting an orientation of the portable device. ### Claim 27 (depends on 26) The computer-readable medium of claim 26, further comprising code for determining an absolute direction of a user directional swipe, irrespective of the orientation of the portable device. ### Claim 28 (depends on 22) The computer-readable medium of claim 22, further comprising code for detecting the user movement with ultrasonic sensors. ### Claim 29 (depends on 22) The computer-readable medium of claim 22, further comprising: code for emitting an ultrasonic signal; code for producing microphone signals in response to the ultrasonic signal; and code for detecting the user movement based on the microphone signals. ### Claim 30 (depends on 29) The computer-readable medium of claim 29, further comprising code for determining the orientation of the portable device based on the microphone signals. ### Claim 31 A method of providing a user interface at an electronic device, comprising: generating, by the electronic device, a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space; the electronic device detecting user movement relative to the electronic device, made in response to the audio cues; and selecting at least one of the audio cues based on the user movement. ### Claim 32 (depends on 31) The method of claim 31, further comprising matching a direction of the user movement to one or more of the audio cues. ### Claim 33 (depends on 31) The method of claim 31, further comprising producing an audio feedback cue based on detected user movement, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement. ### Claim 34 (depends on 31) The method of claim 31, further comprising detecting a directional swipe across a touch screen included in the electronic device. ### Claim 35 (depends on 31) The method of claim 31, further comprising detecting an orientation of the electronic device. ### Claim 36 (depends on 35) The method of claim 35, further comprising determining an absolute direction of a user directional swipe, irrespective of the orientation of the electronic device. ### Claim 37 (depends on 31) The method of claim 31, further comprising detecting the user movement with ultrasonic sensors. ### Claim 38 (depends on 31) The method of claim 31, further comprising: emitting an ultrasonic signal; producing microphone signals in response to the ultrasonic signal; and detecting the user movement based on the microphone signals. ### Claim 39 (depends on 38) The method of claim 38, further comprising determining the orientation of the electronic device based on the microphone signals.