Head pose mixing of audio files
AssigneeMagic Leap, Inc.
InventorsStephen Vincent Mangiat, Michael Benson Tucker, Anastasia Andreyevna Tajik
Life of this patent
Estimating the term…
Abstract
Examples of wearable devices that can present to a user of the display device an audible or visual representation of an audio file comprising a plurality of stem tracks that represent different audio content of the audio file are described. Systems and methods are described that determine the pose of the user; generate, based on the pose of the user, an audio mix of at least one of the plurality of stem tracks of the audio file; generate, based on the pose of the user and the audio mix, a visualization of the audio mix; communicate an audio signal representative of the audio mix to the speaker; and communicate a visual signal representative of the visualization of the audio mix to the display.
Claims19
A computing system comprising:
- a wearable display configured to present, to a user, augmented reality (AR) content in a three-dimensional (3D) environment;
- at least one sensor configured to provide sensor data usable to determine a pose of a user in the 3D environment;
- at least one speaker configured to present audio content to the user;
- and
- at least one processor communicatively coupled to the display, the at least one sensor, and the at least one speaker, the at least one processor configured to execute software instructions to:
- determine, based at least on the sensor data, the pose of the user in the 3D environment;
- cause the wearable display to present a first virtual object and a second virtual object in the 3D environment, wherein each of the first virtual object and the second virtual object includes a respective one or more portions that are each associated with a respective audio track;
- and
- in response to determining that a first portion of the first virtual object and a second portion of the second virtual object are along a direction of the pose of the user in the 3D environment:
- determine a first audio track associated with the first portion of the first virtual object;
- determine a second audio track associated with the second portion of the second virtual object;
- determine an audio mix of at least the first audio track and the second audio track;
- and
- cause the at least one speaker to play the audio mix.
A computer-implemented method, performed by one or more hardware computer processors, the method comprising:
- determining, based at least on sensor data provided by at least one sensor, a pose of a user in a three-dimensional (3D) environment that is presented to the user by a wearable display;
- causing the wearable display to present a first virtual object and a second virtual object in the 3D environment, wherein each of the first virtual object and the second virtual object includes a respective one or more portions that are each associated with a respective audio track;
- and
- in response to determining that a first portion of the first virtual object and a second portion of the second virtual object are along a direction of the pose of the user in the 3D environment:
- determining a first audio track associated with the first portion of the first virtual object;
- determining a second audio track associated with the second portion of the second virtual object;
- determining an audio mix of at least the first audio track and the second audio track;
- and
- causing the audio mix to be played by at least one speaker.
Non-transitory computer-readable storage media storing software instructions that are executable by a computing system to cause the computing system to perform operations comprising:
- determining, based at least on sensor data provided by at least one sensor, a pose of a user in a three-dimensional (3D) environment that is presented to the user by a wearable display;
- causing the wearable display to present a first virtual object and a second virtual object in the 3D environment, wherein each of the first virtual object and the second virtual object includes a respective one or more portions that are each associated with a respective audio track;
- and
- in response to determining that a first portion of the first virtual object and a second portion of the second virtual object are along a direction of the pose of the user in the 3D environment:
- determining a first audio track associated with the first portion of the first virtual object;
- determining a second audio track associated with the second portion of the second virtual object;
- determining an audio mix of at least the first audio track and the second audio track;
- and
- causing the audio mix to be played by at least one speaker.
Dependencies are inferred from the claim text.