# US 12,010,506 B2 · Spatial audio reproduction based on head-to-torso orientation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,506 B2 - Document ID: US12010506B2 - Status: Granted - Kind: B2 - Application number: 17/847,095 - Filed: 2022-06-22 - Published: 2022-10-13 - Granted: 2024-06-11 - Earliest claimed: 2019-09-20 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Antti J. Vanne; Darius A. Satongar; James W. Vandyke; Juha O. Merimaa; Martin E. Johnson; Tomi A. Huttunen - Web page: https://patentgrep.com/patent/US12010506B2 ## CPC - H04S 7/304: {For headphones} - G06F 3/012: {Head tracking input arrangements} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)} - H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} - H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)} - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - H04S: STEREOPHONIC SYSTEMS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract A media system and a method of using the media system to reproduce spatial audio based on head-to-torso orientation, are described. The method includes determining a head-to-source orientation and a head-to-torso orientation based on head orientation data generated by a head tracking device. Determining the head-to-torso orientation includes determining torso movements based on movements of the head. The torso can be determined to move when the head movements meet a head movement condition, such as a predetermined angle of movement or pattern of movement. A binaural audio filter that is based on a head-related transfer function corresponding to both the head-to-source orientation and the head-to-torso orientation is applied to an audio input signal to generate an audio output signal. The audio output signal is played to accurately recreate spatial audio having sounds emitted to the user by a sound source. Other aspects are also described and claimed. ## Claims ### Claim 1 A method, comprising: detecting a head movement of a head of a user based on head orientation data generated by a head mounted device; determining a torso movement of a torso of the user based at least in part on a downward looking camera coupled to the head mounted device, wherein the downward looking camera is arranged to track a pattern of clothing on the torso of the user, wherein determining the torso movement includes identifying and tracking, using the downward looking camera, the pattern of the clothing; determining a head-to-torso orientation based at least in part on the head movement and the torso movement; selecting a head-related transfer function (HRTF) according to the head-to-torso orientation; and applying a binaural audio filter to an audio input signal to generate an audio output signal, wherein the binaural audio filter is based on the HRTF. ### Claim 2 (depends on 1) The method of claim 1, wherein the HRTF takes into account spatial effects of the head-to-torso orientation on spatial audio produced during playback of the audio output signal. ### Claim 3 (depends on 1) The method of claim 1, wherein the torso movement is further based on a forward-facing camera that is a part of a portable device. ### Claim 4 (depends on 1) The method of claim 1 further comprising driving, using the audio output signal, a speaker to reproduce spatial sound that accounts for spatial audio effects of the head-to-torso orientation. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the torso movement is further based on contextual information about one or more of a current state of the user or a current use of the head mounted device that exists when the head movement is detected. ### Claim 6 (depends on 5) The method of claim 5, wherein the contextual information indicates that the user is ambulatory. ### Claim 7 (depends on 5) The method of claim 5, wherein the contextual information indicates that the head mounted device is being used to play audio content of a movie. ### Claim 8 (depends on 1) The method of claim 1, wherein the head movement includes the head moving, and wherein the torso movement includes the torso remaining stationary. ### Claim 9 (depends on 1) The method of claim 1 further comprising playing the audio output signal to recreate spatial audio of sounds. ### Claim 10 A media system, comprising: one or more processors configured to: detect a head movement of a head of a user based on head orientation data generated by a device, determine a torso movement of a torso of the user based at least in part on an image sensor that is arranged to track a pattern of clothing on the torso of the user, wherein the torso movement differs from the head movement, determine a head-to-torso orientation based at least in part on the head movement and the torso movement, select a head-related transfer function (HRTF) according to the head-to-torso orientation, and apply a binaural audio filter to an audio input signal to generate an audio output signal, wherein the binaural audio filter is based on the HRTF; and one or more speakers configured to play the audio output signal to recreate spatial audio of sounds that accounts for spatial audio effects of the head-to-torso orientation. ### Claim 11 (depends on 10) The media system of claim 10, wherein the device is a head mounted device and the image sensor comprises a downward looking camera that is coupled to the head mounted device. ### Claim 12 (depends on 10) The media system of claim 10, wherein the one or more processors are configured to determine the torso movement based on contextual information about one or more of a current state of the user or a current use of the device that exists when the head movement is detected. ### Claim 13 (depends on 10) The media system of claim 10, wherein the head movement includes the head moving, and wherein the torso movement includes the torso remaining stationary. ### Claim 14 A non-transitory computer readable medium containing instructions, which when executed by one or more processors of a media system, cause the media system to perform a method comprising: detecting a head movement of a head of a user based on head orientation data generated by a device; determining a torso movement of a of the user based at least in part on an image sensor that is arranged to track a pattern of clothing on the torso of the user, wherein the torso movement differs from the head movement; determining a head-to-torso orientation based at least in part on the head movement and the torso movement; selecting a head-related transfer function (HRTF) according to the head-to-torso orientation; and applying a binaural audio filter to an audio input signal to generate an audio output signal, wherein the binaural audio filter is based on the HRTF. ### Claim 15 (depends on 14) The non-transitory computer readable medium of claim 14, wherein the device is a head mounted device and the image sensor comprises a downward looking camera that is coupled to the head mounted device. ### Claim 16 (depends on 14) The non-transitory computer readable medium of claim 14, wherein the method further comprises driving, using the audio output signal, a speaker to reproduce spatial sound that accounts for spatial audio effects of the head-to-torso orientation. ### Claim 17 (depends on 14) The non-transitory computer readable medium of claim 14, wherein determining the torso movement is further based on contextual information about one or more of a current state of the user or a current use of the device that exists when the head movement is detected. ### Claim 18 (depends on 14) The non-transitory computer readable medium of claim 14, wherein the head movement includes the head moving, and wherein the torso movement includes the torso remaining stationary.