Immersive augmented reality experiences using spatial audio

A method for use with a device configured to be head mounted on a user, the device comprising a processor, at least one image sensor, and at least one speaker that produces at least three directional audio zones, the method comprising:

• capturing, using the at least one image sensor, images in an environment of the device;
• identifying at least one of an object or feature within the captured images;
• orienting the device with respect to at least one object or feature in the environment by identifying a match between the at least one object or feature in the captured images and at least one stored object or feature in previously obtained information including position information for location points associated with the objects or features,
• determining a position of the device within the environment with respect to a first matched object or feature by identifying at least two location points associated with the first matched object or feature, determining a relative position between the at least two location points associated with the first matched object or feature in the captured images and corresponding location points for the first matched object or feature in the previously obtained obtaining information, and calculating the position of the device from the determined relative position and the position information for the corresponding location points;
• determining a target location within the environment that may be associated with the object or feature;
• determining a current orientation of the device with respect to the target location;
• selectively emitting audio signals from the at least one speaker in respective directional audio zones responsive to the current orientation to guide the user to the target location;
• and
• adjusting a volume of the audio signals to indicate a distance of the device from the target location.

A non-transitory computer-readable medium storing program code for use with a device configured to be head mounted on a user, the device comprising a processor, at least one image sensor, and at least one speaker that produces at least three directional audio zones, the program code, when executed, is operative to cause the processor to:

• capture, using the at least one image sensor, images in an environment of the device;
• identify at least one of an object or feature within the captured images;
• determine a position of the device within the environment with respect to the at least one object or feature;
• orient the device with respect to the at least one object or feature by retrieving previously obtained information from a remote server system, storing the previously obtained information in a memory of the device, identifying a match between the at least one object or feature in the captured images and at least one stored object or feature in the previously obtained information, and determining a position of the device within the environment with respect to the at least one matched object or feature;
• determine a target location within the environment that may be associated with the object or feature;
• determine a current orientation of the device with respect to the target location;
• selectively emit audio signals from the at least one speaker in respective directional audio zones responsive to the current orientation to guide the user to the target location;
• and
• adjust a volume of the audio signals to indicate a distance of the device from the target location.