Generation of audio-visual filters for virtual spaces

A method for using a trained network to generate an audio performance in a volume of space based on visual data, the method comprising the steps of:

• training, by a computer, a network based on input data and a selected training algorithm to generate a trained network, wherein the input data comprises audio data and visual data of events occurring in a plurality of spaces;
• acquiring visual data depicting a physical environment for of the volume of space;
• analyzing, using the trained network, the visual data to identify environmental structural features of the volume of space that correspond to acoustic reflection and absorption behavior of the physical environment;
• generating, based on the identified environmental structural features, a simulated acoustic response profile of the volume of space that includes at least one of reverberation decay time, reflection delay, or impulse response characteristics of the volume of space;
• and
• applying the simulated acoustic response profile to an audio file that is not recorded at the volume of space, such that playback of the audio file is modified to audibly simulate performance within the physical environment of the volume of space.

A system for providing a virtual experience of a performance in a volume of space, the system comprising:

• a processing unit comprising a network trained on historical visual data of a plurality of volumes of space, each of the plurality of volumes of space having features that affect acoustic properties of each of the plurality of volumes of space, wherein the trained network is configured to:
• receive visual data depicting a physical environment for the volume of space, without receiving audio captured from the volume of space;
• extrapolate from the visual data, using the trained network, environmental structural features of the volume of space that correspond to acoustic reflection and absorption behavior of the physical environment;
• generate, based on the extrapolated environmental structural features, a simulated acoustic response profile of the volume of space that includes at least one of reverberation decay time, reflection delay, or impulse response characteristics of the volume of space;
• and
• apply the simulated acoustic response profile to a file comprising audio data such that playback of audio from the file is modified to audibly simulate performance within the physical environment of the volume of space.