Head pose mixing of audio files
AssigneeMagic Leap, Inc.
InventorsStephen Vincent Mangiat, Michael Benson Tucker, Anastasia Andreyevna Tajik
Life of this patent
Estimating the term…
Abstract
Examples of wearable devices that can present to a user of the display device an audible or visual representation of an audio file comprising a plurality of stem tracks that represent different audio content of the audio file are described. Systems and methods are described that determine the pose of the user; generate, based on the pose of the user, an audio mix of at least one of the plurality of stem tracks of the audio file; generate, based on the pose of the user and the audio mix, a visualization of the audio mix; communicate an audio signal representative of the audio mix to the speaker; and communicate a visual signal representative of the visualization of the audio mix to the display.
Claims25
A method for interacting with an augmented reality object, the method comprising, under control of a hardware computer processor:
- generating an augmented reality object for interaction by a wearer of a wearable display system, wherein the augmented reality object includes an interface for modification of a characteristic associated with the augmented reality object and is displayed on a display of the wearable display system, and wherein the augmented reality object is associated with an audio file comprising at least one of a plurality of stem tracks;
- determining a pose of the wearer in a three-dimensional (3D) environment with reference to a plurality of audible regions, wherein each of the audible regions comprises a portion of the 3D environment in which the wearer can perceive one or more of the plurality of stem tracks of the audio file through a speaker of the wearable display system;
- selecting an audible region corresponding to the determined pose of the wearer;
- detecting gestures of the wearer while the wearer interacts with the interface;
- associating the detected gestures with a modification to a characteristic of the augmented reality object presented within the selected audible region;
- modifying the augmented reality object in accordance with the modification to the characteristic of the augmented reality object;
- and
- presenting audiovisual data comprising at least one of the plurality of stem tracks that is associated with the modified augmented reality object and the selected audible region, including playing the at least one of the plurality of stem track via at least the speaker of the wearable display system.
A wearable device comprising:
- non-transitory memory configured to store an audio file comprising a plurality of stem tracks, with each stem track representing different audio content of the audio file;
- a sensor configured to measure information associated with a pose of a user of the wearable device;
- a display configured to present images to an eye of the user of the wearable device;
- a speaker configured to present sounds to the user of the wearable device;
- and
- a processor in communication with the non-transitory memory, the sensor, the speaker, and the display, the processor programmed with executable instructions to:
- generate an augmented reality object for interaction by the user of the wearable device, wherein the augmented reality object includes an interface for modification of a characteristic associated with the augmented reality object and is presented through the display, and wherein the augmented reality object is associated with the audio file comprising at least one of the plurality of stem tracks;
- determine the pose of the wearer in a three-dimensional (3D) environment with reference to a plurality of audible regions, wherein each of the audible regions comprises a portion of the 3D environment in which the wearer can perceive one or more of the plurality of stem tracks of the audio file through a speaker of the wearable display system;
- select an audible region corresponding to the determined pose of the wearer;
- detect gestures of the user while the user interacts with the interface;
- associate the detected gestures with a modification to a characteristic of the augmented reality object presented within the selected audible region;
- modify the augmented reality object in accordance with the modification to the characteristic of the augmented reality object;
- and
- present audiovisual data comprising at least one of the plurality of stem tracks that is associated with the modified augmented reality object and the selected audible region, including playing the at least one of the plurality of stem tracks via at least the speaker of the wearable display system.
Dependencies are inferred from the claim text.