US 12,003,953 B2 · Physics-based audio and haptic synthesis

Abstract

Disclosed herein are systems and methods for generating and presenting virtual audio for mixed reality systems. A method may include determining a collision between a first object and a second object, wherein the first object comprises a first virtual object. A memory storing one or more audio models can be accessed. It can be determined if the one or more audio models stored in the memory comprises an audio model corresponding to the first object. In accordance with a determination that the one or more audio models comprises an audio model corresponding to the first object, an audio signal can be synthesized, wherein the audio signal is based on the collision and the audio model corresponding to the first object, and the audio signal can be presented to a user via a speaker of a head-wearable device. In accordance with a determination that the one or more audio models does not comprise an audio model corresponding to the first object, an acoustic property of the first object can be determined, a custom audio model based on the acoustic property of the first object can be generated, an audio signal can be synthesized, wherein the audio signal is based on the collision and the custom audio model, and the audio signal can be presented, via a speaker of a head-wearable device, to a user.

Claims17

Dependent claims

A method comprising:

presenting, via a transmissive display of a wearable head device, a view of a mixed reality environment;
determining a collision between a first object and a second object in the mixed reality environment, wherein the first object comprises at least one of a first virtual object and a first real object;
accessing a memory storing acoustic data;
determining whether the acoustic data stored in the memory comprises first acoustic data corresponding to the first object;
in accordance with a determination that the acoustic data stored in the memory comprises the first acoustic data corresponding to the first object:
determining a first audio signal based on the collision and based further on the first acoustic data corresponding to the first object, and
concurrently with presenting the view of the mixed reality environment via the transmissive display, presenting the first audio signal via a speaker of the wearable head device;
and
in accordance with a determination that the acoustic data stored in the memory does not comprise the first acoustic data corresponding to the first object:
determining an acoustic property of the first object,
determining a second audio signal based on the collision and based further on the acoustic property, and
concurrently with presenting the view of the mixed reality environment via the transmissive display, presenting the second audio signal via the speaker of the wearable head device.

9 more elements

A system comprising:

a wearable head device comprising:
a transmissive display;
a speaker;
and
one or more sensors;
and
one or more processors configured to perform a method comprising:
presenting, via the transmissive display, a view of a mixed reality environment;
determining a collision between a first object and a second object in the mixed reality environment, wherein the first object comprises at least one of a first virtual object and a first real object;
accessing a memory storing acoustic data;
determining whether the acoustic data stored in the memory comprises first acoustic data corresponding to the first object;
in accordance with a determination that the acoustic data stored in the memory comprises the first acoustic data corresponding to the first object:
determining a first audio signal based on the collision and based further on the first acoustic data corresponding to the first object, and
concurrently with presenting the view of the mixed reality environment via the transmissive display, presenting the first audio signal via the speaker;
and
in accordance with a determination that the acoustic data stored in the memory does not comprise the first acoustic data corresponding to the first object:
determining an acoustic property of the first object,
determining a second audio signal based on the collision and based further on the acoustic property, and
concurrently with presenting the view of the mixed reality environment via the transmissive display, presenting the second audio signal via the speaker.

16 more elements

A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

presenting, via a transmissive display of a wearable head device, a view of a mixed reality environment;
determining a collision between a first object and a second object in the mixed reality environment, wherein the first object comprises one or more of a first virtual object and a first real object;
accessing a memory storing acoustic data;
determining whether the acoustic data stored in the memory comprises first acoustic data corresponding to the first object;
in accordance with a determination that the acoustic data stored in the memory comprises the first acoustic data corresponding to the first object:
determining a first audio signal based on the collision and based further on the first acoustic data corresponding to the first object, and
concurrently with presenting the view of the mixed reality environment via the transmissive display, presenting the first audio signal via a speaker of the wearable head device;
and
in accordance with a determination that the acoustic data stored in the memory does not comprise the first acoustic data corresponding to the first object:
determining an acoustic property of the first object,
determining a second audio signal based on the collision and based further on the acoustic property, and
concurrently with presenting the view of the mixed reality environment via the transmissive display, presenting the second audio signal via the speaker of the wearable head device.

9 more elements

Dependencies are inferred from the claim text.

Physics-based audio and haptic synthesis

Life of this patent

Physics-based audio and haptic synthesis

Life of this patent

Abstract

Claims17