US 2024/0205635 A1 · SYSTEM FOR GENERATING SPATIAL AUDIO WITH UNIFORM REVERBERATION IN REAL-TIME COMMUNICATION

Abstract

A real-time communication software application for generating spatial audio with uniform reverberation in a real-time communication session is performed by an electronic communication device. The application is adapted to remove the reverberation of recorded speech signals from far-end participants by the dereverberation approach, render the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generate reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, and combine direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.

Claims18

Dependent claims

1-17. (canceled)

A real-time communication software application for generating spatial audio with uniform reverberation in a real-time communication session, said real-time communication software application adapted to:

1) retrieve a first stream of audio signals from a first electronic communication device via a network interface of an electronic communication device, wherein said real-time communication software application adapted to be executed by said electronic communication device for generating spatial audio with uniform reverberation in a real-time communication session, wherein said electronic communication device including:
(a) a processing unit;
(b) a memory element operatively coupled to said processing unit;
(c) an audio input interface operatively coupled to said processing unit;
(d) an audio output interface operatively coupled to said processing unit;
(e) a video input interface operatively coupled to said processing unit;
(f) a video output interface operatively coupled to said processing unit;
and
(g) said network interface operatively coupled to said processing unit;
2) remove reverberation from said first stream of audio signals to generate a first stream of dry signals;
3) filter said first stream of dry signals through a head-related transfer function to generate a first direct parts of binaural sounds for a listener;
4) retrieve a second stream of audio signals from a second electronic communication device via said network interface;
5) remove reverberation from said second stream of audio signals to generate a second stream of dry signals;
6) filter said second stream of dry signals through a head-related transfer function to generate a second direct parts of binaural sounds for said listener;
7) sum said first and second streams of dry signals to generate summed dry signals;
8) convolve said summed dry signals with a set of room impulse responses to generate reverberant sound parts of binaural sounds;
9) sum said first and second direct parts of binaural sounds to generate summed direct parts of left ear binaural sounds;
10) sum said first and second direct parts of binaural sounds to generate summed direct parts of right ear binaural sounds;
11) sum said summed direct parts of left ear binaural sounds with said reverberant sound parts of binaural sounds to generate audio signals for said listener's left ear;
and
12) sum said summed direct parts of right ear binaural sounds with said reverberant sound parts of binaural sounds to generate audio signals for said listener's left ear.

18 more elements

1) retrieve a first stream of audio signals from a first electronic communication device via a network interface of an electronic communication device, wherein said real-time communication software application adapted to be executed by said electronic communication device for generating spatial audio with uniform reverberation in a real-time communication session, wherein said electronic communication device including:
(a) a processing unit;
(b) a memory element operatively coupled to said processing unit;
(c) an audio input interface operatively coupled to said processing unit;
(d) an audio output interface operatively coupled to said processing unit;
(e) a video input interface operatively coupled to said processing unit;
(f) a video output interface operatively coupled to said processing unit;
and
(g) said network interface operatively coupled to said processing unit;
2) remove reverberation from said first stream of audio signals to generate a first stream of dry signals;
3) filter said first stream of dry signals through a head-related transfer function to generate a first direct parts of left ear binaural sounds for a listener;
4) retrieve a second stream of audio signals from a second electronic communication device via said network interface;
5) remove reverberation from said second stream of audio signals to generate a second stream of dry signals;
6) filter said second stream of dry signals through a head-related transfer function to generate a second direct parts of right ear binaural sounds for said listener;
7) generate first early reflections for a first far-end participant corresponding to said first electronic communication device;
8) generate second early reflections for a second far-end participant corresponding to said second electronic communication device;
9) sum said first and second streams of dry signals to generate summed dry signals;
10) convolve said summed dry signals with a set of room impulse responses to generate reverberant sound parts of binaural sounds;
11) sum said first and second direct parts of binaural sounds to generate summed direct parts of left ear binaural sounds;
12) sum said first and second direct parts of binaural sounds to generate summed direct parts of right ear binaural sounds;
13) sum said first and second early reflections to generate first summed early reflections for a left ear of said listener;
14) sum said first and second early reflections to generate second summed early reflections for a right ear of said listener;
15) sum said first summed direct parts of left ear binaural sounds and said first summed early reflections for said left ear;
16) sum said first summed direct parts of left ear binaural sounds and said first summed early reflections for said left ear to generate first summed direct parts of binaural sounds with early reflections for said left ear;
17) sum said second summed direct parts of right ear binaural sounds and said second summed early reflections for said right ear to generate second summed direct parts of binaural sounds with early reflections for said right ear;
18) sum said first summed direct parts of binaural sounds with early reflections for said left ear and said reverberant sound parts of binaural sounds to generate audio signals for said left ear;
and
19) sum said second summed direct parts of binaural sounds with early reflections for said right ear and said reverberant sound parts of binaural sounds to generate audio signals for said right ear.

25 more elements

A real-time communication software application for generating spatial audio with uniform reverberation in a real-time communication session for a user with a headset on, said real-time communication software application adapted to:

1) retrieve a stream of audio signals of said user's speech from an audio input interface via a network interface, wherein said real-time communication software application adapted to be executed by said electronic communication device for generating spatial audio with uniform reverberation in a real-time communication session, wherein said electronic communication device including;
(a) a processing unit;
(b) a memory element operatively coupled to said processing unit;
(c) said audio input interface operatively coupled to said processing unit;
(d) an audio output interface operatively coupled to said processing unit;
(e) a video input interface operatively coupled to said processing unit;
(f) a video output interface operatively coupled to said processing unit;
and
(g) said network interface operatively coupled to said processing unit;
2) remove reverberation from said stream of audio signals to generate a stream of dry signals;
3) convolve said stream of dry signals with a set of room impulse responses to generate reverberant sound parts of binaural sounds;
4) filter said reverberant sound parts of binaural sounds through a high-pass filter, thereby forming high-pass filtered signals;
5) filter said high-pass filtered signals through a mouth-ear transfer function, thereby forming mouth-ear transfer function filtered signals;
6) filter said mouth-ear transfer function filtered signals through a set of headphone compensation filters to generate audio signal for said user's left ear;
and
7) filter said mouth-ear transfer function filtered signals through a set of headphone compensation filters to generate audio signal for said user's right ear.

13 more elements

Dependencies are inferred from the claim text.

System for generating spatial audio with uniform reverberation in real-time communication

Life of this patent

System for generating spatial audio with uniform reverberation in real-time communication

Life of this patent

Abstract

Claims18