Deep learning based voice extraction and primary-ambience decomposition for stereo to surround upmixing with dialog-enhanced center channel

A non-transitory processor-readable medium that includes a program that when executed by a processor performs dialog enhancement of extracted sources of an unmixed signal, comprising:

• determining, by the processor, directional sounds from a content mix using a machine learning unmixing model;
• panning, by the processor, the directional sounds in an upmixed signal;
• computing, by the processor, signal-dependent upmixing gains for specific frequency bins on a frame-basis using a machine learning model for the upmixed signal;
• and
• computing, by the processor, dedicated voice clarity gains using a hearing impairment model for a plurality of hearing-impaired profiles for achieving dialog enhancement;
• wherein the signal dependent upmixing gains and voice clarity gains are transmitted as metadata with a downmixed signal representing the content mix.

An apparatus comprising:

• a memory storing instructions;
• and
• at least one processor executes the instructions including a process configured to:
• determine directional sounds from a content mix using a machine learning unmixing model;
• pan the directional sounds in an upmixed signal;
• compute signal-dependent upmixing gains for specific frequency bins on a frame-basis using a machine learning model for the upmixed signal;
• and
• compute dedicated voice clarity gains using a hearing impairment model for a plurality of hearing-impaired profiles for achieving dialog enhancement;
• wherein the signal dependent upmixing gains and voice clarity gains are transmitted as metadata with a downmixed signal representing the content mix.