Back to results
US 11,231,770 B2Granted
Augmented reality system with spatialized audio tied to user manipulated virtual object
AssigneeMagic Leap, Inc.
InventorsBrett Shipes, Marc Alan McCall
Life of this patent
Estimating the term…
Abstract
An augmented reality system includes left and right transparent eyepieces through which a user can view the real world and which serve to couple imagewise modulated light into the user's eyes in order to display virtual content components of a mixed reality experience. The system further includes left and right speakers and a facility for user spatial manipulation of virtual objects. The system produces spatialized audio that has a virtual sound source position fixed to one or more virtual objects that are spatially manipulated by the user. Thus the system provides more realistic visual and auditory presentation of virtual components of a mixed reality experience.
Claims19
A system comprising:
- a handheld controller;
- a wearable head device including:
- a left eye display comprising a left eyepiece;
- a right eye display comprising a right eyepiece;
- a left side speaker;
- and a right side speaker;
- first tracking components;
- and circuitry coupled to the left eye display, the right eye display, the left side speaker, the right side speaker, and the first tracking components, wherein the circuitry is configured to perform a method comprising:
- identifying, via the first tracking components, a position of the handheld controller relative to the wearable head device;
- detecting, via the first tracking components, one or more surfaces of a nearby environment;
- and outputting spatialized audio that includes a left channel audio signal that is output to the left side speaker and a right channel audio signal that is output to the right side speaker, the spatialized audio associated with a virtual sound source location that is substantially fixed relative to the handheld controller, the virtual sound source located a first distance from a first surface of the one or more detected surfaces, wherein:
- the method further comprises displacing a virtual sound source image from the virtual sound source by an amount equal to twice the first distance, and one or more of the left channel audio signal and the right channel audio signal is based on the virtual sound source image that simulates an acoustic reflection off of the first surface of the one or more detected surfaces.
9 more elements
A system comprising:
- a wearable head device including:
- a left eye display comprising an eyepiece;
- a right eye display comprising an eyepiece;
- a left side speaker;
- and a right side speaker;
- a hand tracking machine vision subsystem;
- and circuitry coupled to the left eye display, the right eye display, the left side speaker, the right side speaker, and the hand tracking machine vision subsystem, wherein the circuitry is configured to perform a method comprising:
- identifying, via the hand tracking machine vision subsystem, a position of a hand of a user relative to the wearable head device;
- outputting spatialized audio that includes a left channel audio signal that is output to the left side speaker and a right channel audio signal that is output to the right side speaker, the spatialized audio associated with a virtual sound source location that is substantially fixed relative to the hand of the user, the virtual sound source located a first distance from a first surface of the one or more detected surfaces, wherein:
- the method further comprises displacing a virtual sound source image from the virtual sound source by an amount equal to twice the first distance, and one or more of the left channel audio signal and the right channel audio signal is based on the virtual sound source image that simulates an acoustic reflection off of the first surface of the one or more detected surfaces.
7 more elements
A system comprising:
- a wearable head device including:
- a left eye display comprising an eyepiece;
- a right eye display comprising an eyepiece;
- a left side speaker;
- and a right side speaker;
- one or more receivers;
- and circuitry coupled to the left eye display, the right eye display, the left side speaker, the right side speaker, and the one or more receivers, wherein the circuitry is configured to perform a method comprising:
- determining a location of a virtual object relative to the wearable head device;
- outputting spatialized audio that includes a left channel audio signal that is output to the left side speaker and a right channel audio signal that is output to the right side speaker, the spatialized audio associated with a virtual sound source location that is substantially fixed relative to the virtual object, the virtual sound source located a first distance from a first surface of the one or more detected surfaces;
- and receiving user input through the one or more receivers and in response to receiving the user input through the one or more receivers, change a location of the virtual object and change the at least one virtual sound source location, and wherein:
- the method further comprises displacing a virtual sound source image from the virtual sound source by an amount equal to twice the first distance, and one or more of the left channel audio signal and the right channel audio signal is based on the virtual sound source image that simulates an acoustic reflection off of the first surface of the one or more detected surfaces.
8 more elements
Dependencies are inferred from the claim text.